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(54) Recording medium, apparatus and method of recording data on the same, and apparatus 
and method of reproducing data from the recording medium 



(57) In an apparatus and method of reproducing a 
data unit consisting of a stream of packs of main picture 
data, a data unit consisting of a stream of packs of audio 
data, and a data unit consisting of a stream of packs of 
sub-picture data, which recorded on an optical disk (10) 
and which are reproducible in the same time slot, repro- 
duction data of the data unit including the sub-picture 
data is reproduced is reproduced by a sub-picture 
decoder section (62) after reproduction data of the data 



unit including the audio data and reproduction data of 
the data unit including the video data are simultane- 
ously reproduced by a video decoder section (58) and 
an audio decoder section (60), and after the reproduc- 
tion data of the data unit including the sub-picture, the 
reproduction data of the data unit including the audio 
data and the reproduction data of the data unit including 
the video data are simultaneously reproduced again. 
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Description 

The present invention relates to a recording 
medium, such as an optical disk, for recording com- 
pressed moving picture data, audio data and the like or 
different kinds of data, an apparatus and method of 
recording data on this recording medium, and an appa- 
ratus and method of reproducing data from the same. 

Recently, optical disk players, which can handle 
moving pictures and play optical disks on which picture 
and audio data are digitally recorded, have been devel- 
oped, and are widely used as players for movie soft- 
ware, karaoke and so forth. Further, MPEG (Moving 
Picture Image Coding Expert Group) standards have 
recently become international standards for a data com- 
pression system for moving pictures. This MPEG sys- 
tem performs variable length compression of picture 
data. 

The MPEG2 system is now also being internation- 
ally standardized, so that the system format which 
matches with the MPEG compression system is defined 
as an MPEG2 system layer. This MPEG2 system layer 
defines the setting of a transfer start time and a play- 
back start time, expressed by using a reference time, in 
moving picture data, audio data and other data, so that 
those data can be transferred and reproduced synchro- 
nously. Playback is normally executed using those infor- 
mation. 

As optical disks designed for multifarious uses have 
recently become available, it is not sufficient to consider 
only on the synchronism among individual pieces of 
data. 

For educational purposes, such as learning lan- 
guages, for example, main picture data as video data 
may present questions to users, while sub-picture data 
such as superimposition may give answers to the users. 
In this case, it is significant to show answers after make 
questions. 

Accordingly, it is a primary object of the present 
invention to provide a recording medium on which vari- 
ous types of data, such as main picture data, audio data 
and sub-picture data, constructed in a reproducit>le 
manner, are recorded in such a way that the sub-picture 
data can be reproduced with a predetermined time 
delay with respect to the main picture data. 

It is a secondary object of the present invention to 
provide a method and apparatus of reproducing data 
from a recording medium on which various types of 
data, such as main picture data, audio data and sub-pic- 
ture data, constructed in a reproducible manner, are 
recorded in such a way that the sub-picture data can be 
reproduced with a predetermined time delay with 
respect to the main picture data. 

It is a third object of the present invention to provide 
a method and apparatus of recording various types of 
data, such as main picture data, audio data and sub-pic- 
ture data, constructed in a reproducible manner, in such 
a way that the sub-picture data can be reproduced with 
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a predetermined time delay with respect to the main pic- 
ture data. 

It is a fourth object of the present invention to pro- 
vide a method and apparatus of reproducing various 

5 types of data, such as main picture data, audio data and 
sub-picture data, constructed in a synchronously repro- 
ducible manner, in such a way that the sub-picture data 
can be reproduced with a predetermined time delay, 
preset in a machine, with respect to the main picture 

10 data. 

It is a fifth object of the present invention to provide 
a method and apparatus of reproducing various types of 
data, such as main picture data, audio data and sub-pic- 
ture data, constructed in a synchronously reproducible 
15 manner, in such a way that the sub-picture data is repro- 
duced with a delay time arbitrarily set using a menu 
screen, with respect to the main picture data. 

It is a sixth object of the present invention to provide 
a method and apparatus of reproducing various types of 
20 data, such as main picture data, audio data and sub-pic- 
ture data, constructed in a synchronously reproducible 
manner, in such a way that the sub-picture data is repro- 
duced with a delay time, computed from the difference 
between the display start time given in sub-picture data 
25 for one still picture, and the display end time, with 
respect to the main picture data. 

It is a seventh object of the present invention to pro- 
vide a method and apparatus of displaying pictures in 
such a way that at the time of reproducing superimposi- 
30 tion data or the like as sub-picture data with an arbitrary 
time delay, with respect to main picture data belonging 
to various types of data, such as main picture data, 
audio data and sub-picture data, constructed in a repro- 
ducible manner, the superimposition data or the like 
35 appears as if hidden by a mosaic or filling with black dur- 
ing the delay period. 

To achieve the above object, the present invention 
provides a recording medium comprising a recording 
medium comprising a plurality of data units recorded 
40 thereon, which become objects to be reproduced in a 
time sequential manner, each object being reproduced 
within a given time period; each of the data units being 
constituted of a stream of a plurality of data packs and 
playback time information, the data packs containing 
45 packets of reproduction data including at least one of 
audio data, video data and sub-picture data; the time 
information of that data unit which includes a sub-pic- 
ture being later by a predetermined time than time infor- 
mation of that data unit which Includes audio data or 
50 video data; reproduction data of the data unit including 
the sub-picture being reproduced after reproduction of 
reproduction data of the data unit including the audio 
data or video data. 

The present Invention also provides a recording 
55 apparatus comprising a recording apparatus for record- 
ing reproduction data on a recording medium, compris- 
ing: means for generating a plurality of data packs and 
playback time information, the data packs containing 
packets of reproduction data including at least one of 
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audio data, video data and subiDicture data; means for 
prqjaring a plurality of data units including playback 
time information and having a stream of data packs to 
be reproduced within a given time period as a unit; and 
means for recording the data units on the recording 
medium In a time-sequentially reproducible manner, 
time information of that data unit which includes a sub- 
picture to be reproduced within the given time period 
being later by a predetermined time than time informa- 
tion of that data unit which includes audio data or video 
data to be reproduced within the given time period. 

The present invention further provides a method 
comprising a method of recording reproduction data on 
a recording medium, comprising the steps of: generat- 
ing a plurality of data packs containing packets of repro- 
duction data including at least one of audio data, video 
data and sub-picture data; preparing a plurality of data 
units including playback time information and having a 
stream of data packs to be reproduced within a given 
time period as a unit; and recording the data units on 
the recording medium in a time-sequentially reproduci- 
ble manner, time information of that data unit which 
includes a sub-picture to be reproduced within the given 
time period being later by a predetermined time than 
time information of that data unit which includes audio 
data or video data to be reproduced within the given 
time period. 

The present invention also provides a reproducing 
apparatus comprising a reproducing apparatus for 
reproducing reproduction data from a recording 
medium, comprising: means for acquiring, from the 
recording medium, a plurality of data units recorded 
thereon, which become objects to be reproduced in a 
time sequential manner, each object being reproduced 
witiiin a given time period, each of the data units being 
constituted of a stream of a plurality of data packs and 
playback time information, the data packs containing 
packets of reproduction data including at least one of 
audio data, video data and sub-picture data, time infor- 
mation of that data unit which includes a sub-picture to 
be reproduced within a given time period being later by 
a predetermined time than time information of that data 
unit which includes audio data or video data to be repro- 
duced within the given time period; and means for con- 
verting the data unit of the acquired reproduction data to 
a reproduction signal in accordance with the time infor- 
mation included in the data unit and reproducing the 
reproduction signal, whereby after reproduction data of 
that data unit which includes audio data or video data to 
be reproduced witiiin a given time period is reproduced, 
the reproducing means reproduces reproduction data of 
the data unit including the sub-picture to be reproduced 
within tiie given period. 

The present invention also provides a reproducing 
apparatus comprising a reproducing apparatus for 
reproducing reproduction data from a recording 
medium, comprising: means for acquiring, from the 
recording medium, a plurality of data units recorded 
thereon, which become objects to be reproduced in a 



time sequential manner, each object being reproduced 
within a given time period, each of the data units being 
constituted of a stream of a plurality of data packs and 
playback time information, the data packs containing 

5 packets of reproduction data including at least one of 
audio data, video data and subiDicture data; and means 
for converting the data unit of the acquired reproduction 
data to a reproduction signal in accordance with the 
time information included in the data unit and reproduc- 

10 ing the reproduction signal, whereby after reproduction 
data of that data unit which includes audio data or video 
data to be reproduced within a given time period is 
reproduced, the reproducing means reproduces repro- 
duction data of the data unit including the sub-picture to 

75 be reproduced within the given period when a predeter- 
mined time previously set elapses. 

The present invention also provides a reproducing 
apparatus comprising a reproducing apparatus for 
reproducing reproduction data from a recording 

20 medium, comprising: means for acquiring, from the 
recording medium, a plurality of data units recorded 
thereon, which become objects to be reproduced in a 
time sequential manner, each object being reproduced 
within a given time period, each of the data units being 

25 constituted of a stream of a plurality of data packs and 
playback time information, the data packs containing 
packets of reproduction data including at least one of 
audio data, video data and sul>-picture data; means for 
setting a delay time at which reproduction data of tiiat 

30 data unit which includes a sub-picture to be reproduced 
within a given time period after reproduction data of that 
data unit which includes audio data or video data to be 
reproduced within the given time period is reproduced; 
arxJ means for converting the data unit of the acquired 

35 reproduction data to a reproduction signal in accord- 
ance with the time information included in the data unit 
and reproducing the reproduction signal, whereby after 
reproduction data of that data unit which includes at 
least one of audio data and video data to be reproduced 

40 within a given time period is reproduced, the reproduc- 
ing means reproduces reproduction data of the data unit 
including the sub-picture to be reproduced within the 
given period when the set delay time elapses. 

The present invention also provides a reproducing 

45 apparatus comprising a reproducing apparatus for 
reproducing reproduction data from a recording 
medium, comprising: means for acquiring, from the 
recording medium, a plurality of data units recorded 
thereon, which become objects to be reproduced in a 

50 time sequential manner, each object being reproduced 
within a given time period, each of the data units being 
constituted of a stream of a plurality of data packs and 
playback time information, the data packs containing 
packets of reproduction data including at least one of 

55 audio data, video data and sub-picture data, the time 
information of that data unit which includes a sub-pic- 
ture consisting of a playback start time and playback 
stop time; means for computing a delay time of the data 
unit including tiie sub-picture from tiie playback start 
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time and playback stop time recorded in the data unit 
including the sub-picture of the acquired r^roduction 
data; and means for converting the data unit of the 
acquired reproduction data to a reproduction signal in 
accordance with the time information included in the 
data unit and reproducing the reproduction signal, 
whereby after reproduction data of that data unit which 
includes at least one of audio data and video data to be 
reproduced within a given time period is reproduced, 
the reproducing means reproduces reproduction data of 
the data unit including the sub-picture to be reproduced 
within the given period when the computed delay time 
elapses. 

This Invention can be more fully understood from 
the following detailed description when taken in con- 
junction with the accompanying drawings, in which: 

FIG. 1 Is a schematic block diagram of an optical 
disk apparatus according to the first embodiment of 
the present invention; 

FIG. 2 is a detailed block diagram of the mechanical 
section of the disk drive unit of FIG. 1 ; 
FIG. 3 is a schematic perspective view of the struc- 
ture of an optical disk loaded in the disk drive unit of 
FIG. 1; 

FIG. 4 shows the structure of the logic format of the 
optical disk of FIG. 3: 

FIG. 5 shows the structure of the video manager of 
FIG. 4; 

FIG. 6 shows an example of the structure of the 

video object set (VOBS) shown in FIG. 5; 

FIG. 7 is a diagram to help explain the structure of 

the video object unit of FIG. 6; 

FIG. 8 shows the parameters and contents of the 

volume manger information management table 

(VMGLMAT) in the video manager (VMGI) of FIG. 

5: 

FIG. 9 shows the structure of the title search pointer 
table (TSPT) in the video manager (VMGI) of FIG. 
5; 

FIG. 10 shows the parameters and contents of the 
title search pointer table Information (TSPTI) in the 
title search pointer table (TSPT) of FIG. 9; 
FIG, 1 1 shows the parameters and contents of the 
title search pointer (TT_SRP) corresponding to the 
input number In the title search pointer table (TSPT) 
of FIG. 9; 

FIG. 12 illustrates the structure of the video title set 
of FIG. 4; 

FIG. 13 shows the parameters and contents of the 
video title set information management table 
(VTSLMAT) of the video title set information (VTSI) 
of FIG. 12; 

FIG. 14 illustrates the structure of the video title set 
program chain information table (VTS_PGCIT) of 
the video title set information (VTSI) of FIG. 12; 
FIG. 15 shows the parameters and contents of the 
information (VTS_PGCITI) in the video title set pro- 



gram chain information table (VTS_PGCIT) of FIG. 
14: 

FIG. 16 shows the parameters and contents of the 
search pointer (VTS_PGCIT_SRP) corresponding 

5 to the program chain in the video title set program 
chain information table (VTS_PGCIT) of FIG. 14; 
FIG. 17 illustrates the structure of the program 
chain information (VTS_PGCI) in the video set cor- 
responding to the program chain in the video title 

10 set program chain information table (VTS_PGCIT) 
of FIG. 14; 

FIG. 18 shows the parameters and contents of the 
program chain general information (PGC_GI) In the 
program chain information (VTS_PGCI) of FIG. 17; 

15 FIG. 1 9 illustrates the structure of a program chain 
map (PGC_PGMAP) of the program chain informa- 
tion (VTS_PGCI) of FIG. 1 7; 
FIG. 20 shows the parameters and contents of the 
entry cell numbers (ECELLN) corresponding to the 

20 programs written in the program chain map 
(PGC_PGMAP) of FIG. 19; 

FIG. 21 illustrate the structure of the cell playback 
information table (C_PBIT) of the program chain 
information (VTS_PGCI) of FIG. 17; 

25 FIG. 22 shows the parameters and contents of the 
cell playback information table (C_PBIT) of FIG. 21 ; 
FIG. 23 illustrates the structure of the cell position 
information (C_POSI) in the program chain infor- 
mation (VTS_PGCI) of FIG. 18; 

30 FIG. 24 shows the parameters and contents of the 
cell position information (C_POSI) of FIG. 23; 
FIG. 25 illustrates the structure of the navigation 
pack of FIG. 23; 

FIG. 26 Illustrates the structure of the video, audio, 
35 sub-picture, or VBI pack of FIG. 6; 

FIG. 27 shows the parameters and contents of the 
presentation control information (PCI) in the navi- 
gation pack of FIG. 26; 

FIG. 28 shows the parameters and contents of the 
40 general information (PCI_GI) in the presentation 
control information (PCI) of FIG. 27; 
FIG. 29 shows the parameters and contents of the 
disk search information (DSI) In the navigation pack 
of FIG. 26; 

45 FIG. 30 shows the parameters and contents of the 
DSI general information (DSLGI) in the disk search 
information of FIG. 29; 

FIG. 31 shows the parameters and contents of the 
synchronizing playback information (SYNCI) on the 

so video object (VOB) of FIG. 29; 

FIG. 32 shows the structure of a sub-picture unit; 
FIG. 33 shows parameters and contents of a sub- 
picture unit header (SPUH) of the sub-picture unit 
shown in FIG. 32; 

55 FIG. 34 shows the parameters and contents of a 
display control sequence table (DCSQT) of the sub- 
picture unit shown in FIG. 32; 
FIG. 35 shows the parameters and contents of a 
display control sequence (DCSQ) shown in FIG. 34; 
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FIGS. 36A through 36L present a timing chart for 
explaining the relationship among an input clock, 
output clock, PTS and DTS for use in the reproduc- 
tion of video data, audio data and sub-picture data; 

FIG. 37 illustrates the structure of the video buffer s 
section shown in FIG. 1 ; 

FIG. 38 illustrates the structure of the video 
decoder section shown in FIG. 1 ; 
FIG. 39 illustrates the structure of the audio 
decoder section shown in FIG. 1 ; io 
FIG. 40 illustrates the structure of the sub-picture 
decoder section shown in FIG. 1 ; 
FIG. 41 is a flowchart illustrating the procedures of 
reproducing video data, audio data and sub-picture 
data; is 
FIG. 42 is a flowchart illustrating the detailed repro- 
duction procedures; 

FIGS. 43 A through 43 C show the output states of 
video data, audio data and sub-picture data in the 
reproduction process illustrated in FIG. 41 ; 20 
FIG. 44 is a block diagram schematically showing 
an optical disk apparatus according to the second 
embodiment of this invention; 
FIGS. 45A through 45M present a timing chart for 
explaining the relationship among an input clock, 25 
output clock, PTS and DTS for use in the reproduc- 
tion of video data, audio data and sub-picture data 
according to this second embodiment; 
FIG. 46 is a flowchart illustrating the procedures of 
reproducing video data, audio data and sub-picture 30 
data by the optical disk apparatus shown in FIG. 44; 
FIGS. 47A and 47B show display examples of a 
menu on a key/display section according to the third 
embodiment of this invention; 

FIG. 48 is a flowchart illustrating the procedures of 35 
reproducing video data, audio data and sub-picture 
data according to the third embodiment; 
FIGS. 49A through 49M present a timing chart for 
explaining the relationship among an input clock, 
output clock, PTS and DTS for use in the reproduc- 40 
tion of video data, audio data and sub-picture data 
according to this fourth embodiment; 
FIG. 50 is a flowchart illustrating the procedures of 
reproducing video data, audio data and sub-picture 
data by the optical disk apparatus shown in FIG. 48; 45 
FIGS. 51 A through 51 C show the output states of 
video data, audio data and sub-picture data at the 
time superimposition is displayed after a phrase; 
FIGS. 52A through 52C show the output states of 
video data, audio data and sub-picture data at the so 
time of displaying superimpositiqn before freezing 
the video and audio; 

FIGS. 53 A through 53C show the output states of 
video data, audio data and sub-picture data at the 
time sounds for only a phrase portion are repro- ss 
duced with some delay and the video image and 
sub-picture are frozen; 

FIGS. 54A through 54C show the output states of 
video data, audio data and sub-picture data at the 



time the second sound is reproduced with some 
delay and the video image, sub-picture and first 
sound are frozen: 

FIGS. 55A through 55C show the output states of 
video data, audio data and sub-picture data at the 
time the first sound and sub-picture are reproduced 
with some delay and the video image and second 
sound are frozen: 

FIGS. 56A through 56C show the output states of 
video data, audio data and sub-picture data at the 
time the second sub-picture is reproduced with 
some delay and the video image, sounds and the 
first sub-picture are frozen; 

FIG. 57 is a block diagram schematically showing 
an optical disk apparatus according to another 
embodiment; 

FIGS. 58A and 5dB show the output states of video 
data, audio data and sub-picture data at the time of 
presenting the mosaic display of sub-pictures for a 
delay time; 

FIG. 59 is a block diagram of an encoder system 
that encodes the video data and generates a video 
file; 

FIG. 60 is a flowchart for an encoding process of 
FIG. 59; 

FIG. 61 is a flowchart for combining the main video 
data, audio data, and sup-picture data all encoded 
according to the flow of FIG. 60 to create a video 
data file; 

FIG. 62 is a block diagram of a disk formatter sys- 
tem that records the formatted video files on an 
optical disk; 

FIG. 63 is a flowchart for creating logic data to be 
recorded on a disk in the disk formatter of FIG. 62; 
FIG. 64 is a flowchart for creating from the logic 
data the physical data to be recorded on a disk; and 
FIG. 65 is a schematic diagram of a system that 
transfers the video title set of FIG. 4 via a communi- 
cation route. 

Hereinafter, referring to FIGS. 1 through 43C, an 
optical disk reproducing apparatus according to the first 
embodiment of the present invention will be explained. 

FIG. 1 is a block diagram of an optical disk repro- 
ducing apparatus that reproduces the data from an opti- 
cal disk associated with an embodiment of the present 
invention. FIG. 2 is a block diagram of the disk drive sec- 
tion that drives the optical disk shown in FIG. 1 . FIG. 3 
shows the structure of the optical disk shown in FIGS. 1 
and 2. 

As shown in FIG. 1. the optical disk reproducing 
apparatus comprises a key/display section 4, a monitor 
section 6, and a speaker section 8. When the user oper- 
ates the key/display section 4. this causes the recorded 
data to be reproduced from an optical disk 10. The 
recorded data contains video data, sub-picture data, 
and audio data, which are converted into video signals 
and audio signals. The monitor section 6 displays 
images according to the audio signals and the speaker 
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section 8 generates sound according to the audio sig- 
nals. 

It Is known that the optical disk 10 is available with 
various structures. For instance, one type of the optical 
disk 10 is a read-only disk on which data is recorded 5 
with a high density as shown in FIG. 3. The optical disk 
10, as shown In FIG. 3, is made up of a pair of compos- 
ite layers 18 and an adhesive layer 20 sandwiched 
between the composite disk layers 18. Each of the com- 
posite disk layers 18 is composed of a transparent sub- io 
strate 14 and a recording layer or a light-reflecting layer 
16. The disk layer 18 is arranged so that the light-reflect- 
ing layer 16 may be in contact with the surface of the 
adhesive layer 20. A center hole 22 is made In the opti- 
cal disk 10. On the periphery of the center hole 22 on is 
both sides, clamping areas 24 are provided which are 
used to clamp the optical disk 10 during its rotation. 
When the disk 10 is loaded in the optical disk apparatus, 
the spindle of a spindle motor 12 shown in FIG. 2 is 
inserted into the center hole 22. As long as the disk is 20 
being rotated, it continues clamped at the clamping 
areas 24. 

As shown in FIG. 3. the optical disk 10 has an Infor- 
mation zone 25 around the clamping zone 24 on each 
side, the information zones allowing the information to 25 
be recorded on the optical disk 10. In each Information 
area 25. its outer circumference area is determined to 
be a lead-out area 26 in which no information is nor- 
mally recorded, its inner circumference area adjoining 
the clamping area 24 Is determined to be a lead-in area 30 
27 in which no information is normally recorded, and the 
area between the lead-out area 26 and the lead-in area 
27 is determined to be a data recording area 28. 

At the recording layer 16 in the Information area 25, 
a continuous spiral track Is normally formed as an area 35 
in which data is to be recorded. The continuous track is 
divided into a plurality of physical sectors, which are 
assigned serial numbers. On the basis of the sectors, 
data is recorded. The data recording area 28 in the 
information recording area 25 Is an actual data record- 40 
ing area, in which management data, main video data, 
sub-picture data, and audio data are recorded In the 
form of pits (that is, in the form of changes in the physi- 
cal state) as explained later. With the read-only optical 
disk 1 0. a train of pits is previously formed in the trans- 45 
parent substrate 14 by a stamper, a reflecting layer is 
formed by evaporation on the surface of the transparent 
substrate 14 in which the pit train is formed, and the 
reflecting layer serves as the recording layer 16. In the 
read-only optical disk 10, a groove is normally not pro- so 
vided as a track and the pit train in the surface of the 
transparent substrate 14 serves as a track. 

The optical disk apparatus 12, as shown in FIG. 1, 
further comprises a disk drive section 30, a system CPU 
section 50, a system ROM/RAM section 52. a system ss 
processor section 54, a data RAM section 56, a buffer 
control STC control section 57, a video decoder 58, a 
video buffer section 59, an audio decoder section 60. an 
audio buffer section 61, a sub-picture decoder section 



62, a sub-picture buffer section 63, a mixing section 64. 
a video D/A section 66 and an audio D/A section 68. 

As shown in FIG. 2, the disk drive section 30 con- 
tains a motor driving circuit 11, a spindle motor 12. an 
optical head 32 (i.e., an optical pickup), a feed motor 33, 
a focus circuit 36. a feed motor driving circuit 37, a track- 
ing drcuit 38, a head amplifier 40, and a servo process- 
ing circuit 44. The optical disk 10 is placed on the 
spindle motor 12 driven by the motor driving circuit 11 
and is rotated by the spindle motor 1 2. The optical head 
32 that projects a laser beam on the optical disk 10 is 
located under the optical disk 10. The optical head 32 is 
placed on a guide mechanism (not shown). The feed 
motor driving circuit 37 is provided to supply a driving 
signal to the feed motor 33. The motor 33 is driven by 
the driving signal and moves in and out the optical head 
32 across the radius of the optical disk 1 0. The optical 
head 32 Is provided with an object lens 34 positioned so 
as to face the optical disk 10. The object lens 34 is 
moved according to the driving signal supplied from the 
focus circuit 36 so as to move along its optical axis. 

To reproduce the data from the above optical disk, 
the optical head 32 projects a laser beam on the optical 
disk 10 via the object lens 34. The object lens 34 is 
moved little by little across the radius of the optical disk 
10 according to the driving signal supplied from the 
tracking circuit 38. Furthermore, the object lens 34 is 
moved along its optical axis according to the driving sig- 
nal supplied from the focusing circuit 36 so that its focal 
point may be positioned on the recording layer 16 of the 
optical disk 10. This causes the laser beam to form the 
smallest beam spot on the spiral track (i.e., the pit train), 
enabling the beam spot to trace the track. The laser 
beam is reflected from the recording layer 16 and 
returned to the optical head 32. The optical head 32 
converts the beam reflected from the optical disk 10 into 
an electric signal, which is supplied from the optical 
head 32 to the servo processing drcuit 44 via the head 
amplifier 40. From the electric signal, the servo process- 
ing circuit 44 produces a focus signal, a tracking signal, 
and a motor control signal and supplies these signals to 
the focus circuit 36, tracking circuit 38. and motor driving 
circuit 1 1 , respectively 

Therefore, the object lens 34 is moved along its 
optical axis and across the radius of the optical disk 10, 
Its focal point Is positioned on the recording layer 16 of 
the optical disk 10. and the laser beam forms the small- 
est beam spot on the spiral track. Furthermore, the spin- 
dle motor 12 is rotated by the motor driving circuit 1 1 at 
a specific rotating speed. This allows the beam to track 
at. for example, a constant linear speed. 

The system CPU section 50 of FIG. 1 supplies to 
the servo processing circuit 44 a control signal serving 
as an access signal. In response to the control signal, 
the servo processing circuit 44 supplies a head-moving 
signal to the feed motor driving circuit 37. which sup- 
plies a driving signal to the feed motor 33. Then, the 
feed motor 33 is driven, causing the optical head 32 to 
move across the radius of the optical disk 10. Then, the 
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optical head 32 accesses a specific sector formed at the 
recording layer 16 of the optica! disk 10. The data is 
reproduced from the specific sector by the optical head 
32, which then supplies it to the head amplifier 40. The 
head amplifier 40 amplifies the reproduced data, which 
is outputted at the disk drive section 30. 

The reproduced data Is transferred and stored in a 
data RAM section 56 by the system processor section 
54 which is controlled by the system CPU section 50 
which is operated in accordance with the programs 
stored in the system ROM/RAM section 52. The stored 
reproduced data is processed at the system processor 
section 54. which sorts the data into video data, audio 
data, and sub-picture data. The video data is sent via 
the video buffer section 59 to the video decoder section 
58 to be decoded, the audio data is sent via the audio 
buffer section 61 to the audio decoder section 66 to be 
decoded, and the sub-picture data is sent via the sub- 
picture buffer section 63 to the sub-picture decoder sec- 
tion 62 to be decoded. The decoded video data and 
sub-picture data are mixed by the mixing section 64, 
and the resultant data is converted to an analog video 
signal by the video D/A section 66. This analog video 
signal is then supplied to the monitor section 6. The 
audio data is converted by the audio D/A section 68 to 
an audio signal which is supplied to the speaker section 
8. Then, on the basis of the video signal, an image is 
displayed on the monitor section 6 and according to the 
audio signal, sound is simultaneously reproduced from 
the speaker section 8. 

The detailed operation of the optical disk apparatus 
of FIG. 1 will be described later with reference to the 
logic format of the optical disk explained below. 

The data recording area 28 between the lead-in 
area 27 and the lead-out area 26 on the optical disk of 
FIG. 1 has a volume and file structure as shown in FIG. 
4. The structure has been determined in conformity to 
specific logic format standards, such as Micro UDF or 
ISO 9660. The data recording area 28 is physically 
divided into a plurality of sectors as described earlier. 
These physical sectors are assigned serial numbers. In 
the following explanation, a logical address means a 
logical sector number (LSN) as determined in Micro 
UDF or ISO 9660. Like a physical sector, a logical sector 
contains 2048 bytes. The numbers (LSN) of logical sec- 
tors are assigned consecutively in ascending order as 
the physical sector number increments. 

As shown in FIG. 4, the volume and file structure is 
a hierarchic structure and contains a volume artd file 
structure area 70, a video manager 71, at least one 
video title set #i 72, and another recorded area 73. 
These areas 70. 71, 72, 73 are aligned with the bound- 
aries between logical sectors. As with a conventional 
CD. a logical sector is defined as a set of 2048 bytes. 
Accordingly, one logical sector corresponds to one logi- 
cal block. 

The volume and file structure area 70 conresponds 
to a management area determined in Micro UDF or ISO 
9660. According to the description in the management 



area, the video manager 71 is stored in the system 
ROM/RAM section 52. As explained with reference to 
FIG. 5. the information used to manage video title sets 
72 is written in the video manager 71, which is com- 

5 posed of a plurality of files, starting with file #0. In each 
video title set 72, compressed video data, compressed 
audio data, compressed sub-picture data, and the 
reproducing information about these data items are 
stored as explained later. Each video title set is com- 

10 posed of a plurality of files 74. The number of video title 
sets 72 is limited to a maximum of 99. Furthermore, the 
number of files 74 (from file #j to file #j+9) constituting 
each video title set 72 is determined to be a maximum 
of 10. These files 74 are also aligned with the bounda- 

75 ries between logical sectors. 

In the other recorded area 73, the information capa- 
ble of utilizing the aforementioned video title sets 72 is 
recorded. The other recorded areas 73 are not neces- 
sarily provided. 

20 As shown in FIG. 5, the video manager 71 contains 
at least three items each corresponding to individual 
files 74. Specifically, the video manager 71 is made up 
of volume manager information (VMGI) 75, a video 
object set for video manager menu (VMGM_VOBS) 76. 

25 and backup of video manager information (VMGLBUP) 
77. Here, the volume manager information (VMGI) 75 
and the backup of video manager information 
(VMGLBUP) 77 are determined to be indispensable 
items, and the video object set for video manager menu 

30 (VMGM_VOBS) 76 is determined to be an optional 
item. In the video object set 76 for VMGM. the video 
data, audio data, and sup-picture data about a menu of 
the volumes of the optical disk managed by the video 
manager 71 are stored. 

35 By the video object set 76 for VMGM 
(VMGM_VOBS), the volume name of the optical disk 
10. the sound accompanying the volume name repre- 
sentation, and the description of the sub-picture are dis- 
played and at the same time, selectable items are 

40 provided in the form of sub-pictures as in video repro- 
duction explained later. For example, the video object 
set 76 for VMGM (VMGM_VOBS) indicates that the 
optical disk 10 contains the video data representing 
English conversation for learning English. Specifically, 

45 lessons on English conversation, together with a vol- 
ume name, such as the contents of English conversa- 
tion, is reproduced in the form of video data and at the 
same time, a theme song is reproduced in sound, and 
the level of the teaching material is displayed in a sup- 
so picture. Furthermore, the user is asked which one of 
lessens 1 to N to select. Thus, for example, the VMGM 
video object set 76 (VMGM_VOBS) provides the user 
with preparations to watch an English conversation pro- 
gram for learning English. 

55 Here, the structure of a video object set (VOBS) 82 
will be described with reference to FIG. 6. FIG. 6 shows 
an example of a video object set (VOBS) 82. The video 
object set (VOBS) 82 comes in three types for two 
menus and a title. Specifically, the video object set 
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(VOBS) 82 contains the video title set for video manager 
menu {VMGM_VOBS) 76, a video object set 
(VTSM_VOBS) 95 for a menu in the video title sets 72 
and a video object set (VTSTT^VOBS) 82 for the titles 
of at least one video title set 72 in a video title set (VTS) 5 
72 as explained later. Each video object set 82 has the 
same structure except that their uses differ. 

As shown in FIG. 6. a video object set (VOBS) 82 is 
defined as a set of one or more video objects (VOB) 83. 
The video objects 83 in a video object set (VOBS) 82 is 10 
used for the same application. A video object set 
(VOBS) 82 for menus is usually made up of one video 
object (VOB) 83 and stores the data used to display a 
plurality of menu screens. In contrast, a video object set 
(VTSTT_VOBS) 82 for title sets is usually composed of 15 
a plurality of video objects (VOB) 83. 

When the aforementioned video of an English con- 
versation program for learning English is taken as 
example, a video object (VOB) 83 corresponds to the 
video data of each lessen. Specifying a particular video 20 
object (VOB) 83 enables, for exanple. lesson 1 to be 
reproduced on a video. The video object set 
(VTSM_VOBS) 95 for a menu of the video title sets 72 
contains the menu data for the English conversation. 
According to the presentation of the menu, a particular 25 
lesson, for example, lesson 1 , can be specified. In the 
case of a usual single story movie, one video object 
(VOB) 83 corresponds to one video object set (VOBS) 
82. One video stream is completed with one video 
object set (VOBS) 82. In the case of a collection of ani- 30 
mated cartoons or an omnibus movie, a plurality of 
video streams each corresponding to individual stories 
are provided in a single video object set (VOBS) 82. 
Each video stream Is stored in the corresponding video 
object 83. Accordingly, the audio stream and sub-pic- 35 
ture stream related to the video stream are also com- 
pleted with each video object (VOB) 83. 

An identification number (IDN#j) is assigned to a 
video object (VOB) 83. By the identification number 
(ID^^). the video object (VOB) 83 can be identified. A 40 
video object (VOB) 83 further comprises one or more 
such cells 84. Although a usual video stream is made up 
of a plurality of cells 84, a menu video stream, or a video 
object (VOB) 83 may be composed of one cell 84. A cell 
84 is likewise assigned an identification number 45 
(C_IDN#j). By the identification number (G_IDN#j). the 
cell 84 is identified. 

As shown in FIG, 6, each cell 84 is composed of 
one or more video object units (VOBU) 85, normally a 
plurality of video object units (VOBU) 85. Here, a video so 
object unit (VOBU) 85 is defined as a pack train having 
a navigation pack (NAV pack) 86 at its head. Specifi- 
cally, a video object unit (VOBU) 85 is defined as a set 
of all the packs recorded, starting at a navigation pack 
(NAV pack) 86 to immediately in front of the next naviga- 55 
tion pack. The reproducing time of the video object unit 
(VOBU) 85 corresponds to the reproducing time of the 
video data made up of one or more GOP (Group of Pic- 
tures) contained in the video object unit (VOBU) 85 as 



shown in FIG. 6. The maximum reproducing time is 
determined to be 0.4 or more second and less than one 
second. In MPEG, one GOP is defined as compressed 
image data which corresponds to about 1 5 frames to be 
played back for, usually, 0.5 seconds. 

In the case where the video data includes a video 
object unit as illustrated in FIG. 6, GOPs each com- 
posed of a video pack (V pack) 88. a sub-picture pack 
(SP pack) 90 and an audio pack (A pack) 91 . all comply- 
ing to the MPEG standards, are arranged, producing a 
vide data stream. Regardless of the number of GOPs, a 
video object (VOBU) 83 Is determined based on the 
time required for producing a GOP. In the video object 
(VOBU) 83 has a navigation pack (NV) pack 86 
arranged at its head. Any reproduced data, whether 
audio data, sub-picture data, or audio/sub-picture data, 
consists of one more video object units. Thus, even if a 
video object unit is composed of only audio packs only, 
these audio packs will be reproduced within the time for 
reproducing the video object unit. The procedure of 
reproducing these packs will be explained later in detail, 
along with the procedure of reproducing the navigation 
pack (NV pack) 86. 

The procedure for reproducing these packs, 
together with the navigation pack, will be explained in 
detail later. 

The video manager 71 will be explained with refer- 
ence to FIG- 5. The volume management information 75 
placed at the head of the video manager 71 contains the 
information used to search for titles and the information 
used to manage the video title sets (VTS) 72 such as 
the information used for reproduction of a video man- 
ager menu. The volume management information 75 
contains at least three tables 78, 79. 80 in the order 
shown in FIG. 5. Each of these tables 78, 79, 80 is 
aligned with the boundaries between logical sectors. A 
first table 78, a volume manger information manage- 
ment table (VMGI_MAT) 78 is a mandatory table, in 
which the size of the video manager 71 , the starting 
address of each piece of the Information In the video 
manger 71, and the attribute information about the 
video object set (VMGM_VOBS) 76 for a video manager 
menu are written. 

Written in a second table of the video manager 71 , 
a title search pointer table 79 (TT_SRPT). is an entry 
program chain (EPGC) of the video titles (VTS) 72 con- 
tained in the optical disk 10 that are selectable accord- 
ing to the entry of a title number from the key/display 
section 4 on the apparatus. 

Here, a program chain 87 is a set of programs 89 
that reproduce the story of a title as shown in FIG. 7. 
Continuous reproduction of a program chain 87 com- 
pletes an educational program such as English conver- 
sation for learning English of a title. Therefore, the user 
can start to see the English conversation from a partic- 
ular scene of the conversation by specifying the pro- 
gram 89. 

In a third table of the video manager 71 . a video title 
set attribution table (VTS_ATRT) 80. the attribute infor- 
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mation determined in the video title set 72 in the volume 
of the optical disk 10 is written. Specifically, In this table 
72, the following items are written as attribute informa- 
tion: the nunnber of video title sets, video title set num- 
bers, video attributes, such as a video data 
compression scheme, audio stream attributes, such as 
an audio coding mode, and subiDicture attributes, such 
as the type of sup-picture display. 

The contents of the volume manager information 
management table (VMGI_MAT) 78 and the title search 
pointer table (TT_SRPT) 79 will be explained in detail 
with reference to FIGS. 8, 9, 10. and 1 1 . 

As shown in FIG. 8, written in the volume manager 
information management table (VMGI_MAT) 78 are a 
video manager identifier (VMG_!D), the size of video 
manager information (VMGLSZ) in logical blocks (one 
logical block contains 2048 bytes as mentioned above), 
the version number (VERN) of the optical disk 10 com- 
plying with the digital versatile disk (DVD) video specifi- 
cation, and the category of video manager 71 
(VMG_CAT). 

In the category of video manager 71 (VMG_CAT), a 
flag indicating whether or not the DVD video directory 
prohibits copying is written. Further written in the table 
(VMGLMAT) are a volume set identifier (VLMSJD), the 
number of video title sets (VTS_Ns). the identifier for a 
person supplying the data to be recorded on the disk 
(Provider Unique ID: PVR_ID). a video manager menu 
video object set start address (VNGM_VOBS_SA). the 
end address (VMGLMAT_EA) of a volume manager 
information management table (VMGI-MAT) 78. and the 
start address (TT_SRPT_SA) of a title search pointer 
table (TT_SRPT) 79. The end address of VMG_MAT 78 
(VMGLMAT_EA) and the start address of TT_SRPT 
(TT_SRPT_SA) 79 are represented by the number of 
logical blocks, relative to the first logical block. 

Furthermore, in the table 78. the start address 
(VTS_ATRT_SA) of the attribute table (VTS_ATRT) 80 
of video title sets (VTS) 72 is represented by the 
number of bytes, relative to the first byte in the VMGI 
manager table (VMGI_MAT) 71, and the video attribute 
(VMGM_V_AST) of the video manager menu (VMGM) 
is written. Further written in the table 78 are the number 
of audio streams (VMGM_AST_Ns) in the video man- 
ager menu (VfVIGM). the attributes of audio streams 
(VMGM_AST_ATR) in the video manager menu 
(VMGM), the number of sub-picture streams 
(VMGM_SPST_Ns) in the video manager menu 
(VMGM), and the attributes of sub-picture streams 
(VMGM_SPST_ATR) in the video manager menu 
(VMGM). 

In the title search pointer table (TT_SRPT) 79. as 
shown in FIG. 9, the title search pointer table Informa- 
tion (TSPTI) is first written and then as many title search 
pointers for input numbers 1 to n (n ^ 99) as are needed 
are written consecutively. When only the reproducing 
data for one title, for example, only the video data for 
one title, is stored in the volume of the optical disK only 



one title search pointer (TT_SRP) 93 is written in the 
table (TT_SRPT) 79. 

The title search pointer table Information (TSPTI) 
92 contains the number of entry program chains 

5 (EN_PGC_Ns) and the end address (TT_SRPT_EA) of 
the title search pointer 93 as shown in FIG. 10. The 
address (TT_SRPT_EA) Is represented by the number 
of bytes, relative to the first byte in the title search 
pointer table (TT_SRPT) 79. Furthermore, as shown In 

10 FIG. 1 1 . each title search pointer (TT_SRP) 93 contains 
the video title set number (VTSN). the program chain 
number (PGCN), and the start address (VTS_SA) of the 
video title set 72. The contents of the title search pointer 
(TT_SRP) 93 specifies a video title set 72 to be repro- 

75 duced and a program chain (PGC) 87 as well as a loca- 
tion in which the video title set 72 is to be stored. The 
start address (VTS_SA) of the video title set 72 is repre- 
sented by the number of logical blocks in connection 
with the title set 72 specified by the video title set 

20 number (VTSN). 

Jhe structure of the logic format of the video title set 
72 shown In FIG. 4 will be described with reference to 
FIG. 12. In each video title set (VTS) 72, four items are 
written in the order shown In FIG. 12. Each video title 

25 set (VTS) 72 is made up of one or more video titles hav- 
ing common attributes. The video title set information 
(VTSI) 94 contains the management information on the 
video titles, including the information on erttry search 
points, the information on playback of video object sets 

30 72. the information on playback of title set menus 
(VTSM), and the attribute information on video object 
sets 72. 

Each video title set 72 is provided with the backup 
of the video title set information (VTSLBUP) 97. 

35 Between the video title set information (VTSI) and the 
backup of the Information (VTS1_BUP) 97. a video 
object set for video title set menus (VTSM_VOBS) 95 
and a video object set for video title set titles 
(VTSTT_VOBS) 96 are arranged. Both of the video 

40 Object sets (VTSM_VOBS and VTSTT_VOBS) 95, 96 
have the structure shown in FIG. 6. as explained earlier. 

"The video title set information (VTSI) 94. the 
backup of the Information (VTSLBUP) 97, and the 
video object set (VTSTT_VOBS) 96 for video title set 

45 titles are items Indispensable to the video title set 72. 
The video object set (VTSM_VOBS) 96 for video title set 
menus is an option provided as the need arises. 

The video title set Information (VTSI) 94 consists of 
four tables as shown in FIG. 12. The four tables are 

so forced to align with the boundaries between logical sec- 
tors. The video title set information management table 
(VTSLMAT) 98. a first table, is a mandatory table. In 
which the size of the video title set (VTS) 72. the start 
addresses of each piece of information in the video title 

55 set (VTS) 72. and the attributes of the video object sets 
(VOBS) 96 in the video title set (VTS) 72 are written. 

The video title set direct access pointer table 
(VTS_DAPT) 99. a second table, is an optional table 
provided as the need arises. Written in the table 99 are 
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the program chains (PGC) 87 and/or programs (PG) 89 
contained in the video title set 72 that are selectable 
according to the entry of a title number from the key/dis- 
play section 4 of the apparatus. 

The video title set program chain information table s 
(VTS_PGCIT) 100. a third table, is a mandatory table, in 
which the VTS program chain information (VTS_PGCI) 
104 is written. The video title set time search map table 
(VTS_MAPT) 101. a fourth table, is an optional table 
provided as the need arises, in which the information on io 
the recording location, for a specific period of time of 
display, of the video data in each program chain (PGC) 
87 in the title set 72 to which the map table 
(VTS_MAPT) 101 belongs is written. 

Next, the video title information management table is 
(VTSLMAT) 98 and video title set program chain infor- 
mation table (VTSLPGCIT) 100 shown in FIG. 12 will 
be described with reference to FIGS. 13 to 20. 

FIG. 13 shows the contents of the video title infor- 
mation management table (VTSLMAT) 98, in which the 20 
video title set identifier (VTS_ID). the size of the video 
title set 72 (VTS_SZ), the version number of the DVD 
video specification (VERN), and the video title set cate- 
gory (VTS_CAT) are written in that order. Furthermore, 
in the table (VTSLMAT) 98, the start address of the 25 
video object set (VTSM_VOBS) for VTS menus is 
expressed by a logical block relative to the first logical 
block in the video title set (VTS) 72. The start address 
(VTSTT_VOB_SA) of the video object 83 for titles in the 
video title set (VTS) 72 is expressed by a logical block 30 
(RLBN) relative to the first logical block in the video title 
set (VTS) 72. 

Furthermore, in the table (VTSLMAT) 98, the end 
address (VTLMAT_EA) of the video title set information 
management table (VTLM AT) 98 is represented by the 35 
number of blocks, relative to the first byte in the table 
(VTLMAT), 98 and the start address of the video title 
set direct access pointer table (VTS_DAPT) 99 is repre- 
sented by the number of blocks, relative to the first byte 
in the video title set information (VTSI) 94. 40 

Still furthermore, in the table (VTSLMAT) 98, the 
start address (VTS_PGCIT_SA) of the video title set 
program chain information table (PGCIT) 100 is 
expressed by the number of blocks, relative to the first 
byte of the video title set information (VTSI) 94. and the 45 
start address (VTS_MAPT_SA) of the time search map 
(VTS_MAPT) 101 in the video title set (VTS) 72 is 
expressed by a logical sector relative to the first logical 
sector in the video title set (VTS) 72. Written in the table 
(VTSLMAT) 98 are the video object set (VTSM_VOBS) so 
95 for the video title set menu (VTSM) in the video title 
set (VTS) 72. the video attributes (VTS„V_ATR) of the 
video object set (VTST_VOBS) 96 for the titles (VTSTT) 
in the video title set 72, and the number of audio 
streams (VTS_AST_Ns) in the video object set ss 
(VTSTT_VOBS) 82 for the titles (VTSTT) for the video 
title sets 72. Here, the video attributes (VTS_V_ATR) 
include a video compression mode, a frame rate of the 



TV system, and an aspect ratio for displaying an image 
on a display unit. 

Written in the table (VTSLMAT) 98 are the audio 
stream attributes (VTS_AST_ATR) of the video object 
set (VTST_VOBS) for the titles (VTSTT) in the video title 
set (VTS). The attributes (VTS_AST_ATR) include an 
audio encoding mode indicating how audio is encoded, 
the number of bits used to quantize audio, and the 
number of audio channels. 

Furthermore, written in the table (VTSLMAT) 98 
are the number (VTS_SPST_Ns) of sub-picture 
streams in the video object set 82 for the titles (VTSTT) 
in the video title set (VTS) 72 and the attributes 
(VTS_SPST_ATR) of each sub-picture stream. The 
attributes (VTR_SPST_ATR) for each sub-picture 
stream include a sub-picture coding mode and a sub- 
picture display type. 

Furthermore, in the table (VTSLMAT) 98, written 
are the number (VTSM_AST_Ns) of audio streams in 
the video title set menu (VTSM), the audio stream 
attributes (VTSM_AST__ATR), the number 
(VTSM_SPST_Ns) of sup-picture streams, and the sup- 
picture stream attributes (VISM_SPST_ATR). 

The VTS program chain information table 
(VTS_PGCIT) 100 has a structure as shown in FIG. 14. 
In the information table (VTS_PGCIT) 100. information 
on the VTS program chain (VTS^PGC) is written, start- 
ing with information (VTS_PGCIT_I) 102 on the infor- 
mation table (VTS_PGCIT) 100 related to the VTS 
program chain (VTS_PGC). In the information table 
(VTS_PGCIT) 100. the information (VTS„PGCITJ) 102 
is followed by as many VTS_PGCI search pointer 
(VTS_PGCIT_SRP) 103 used to search for VTS pro- 
gram chains (VTS_PGC) as the number (#1 to #n) of 
VTS program chains in the information table 
(VTS_PGCIT). At the end of the table, there are pro- 
vided as many pieces of information (VTS_PGGI) 104 
on the respective VTS program chains (VTS_PGC) as 
the number (from #1 to #n) of the VTS program chains 
(VTS_PGC), 

The information (VTS_PGGIT_I) 102 In the VTS 
program chain Information table (VTS_PGCIT) 100 con- 
tains the number (VTS_PGC_Ns) of VTS program 
chains {VTS_PGC) as shown in FIG. 15. The end 
address (VTS_PGCIT_EA) of the table information 
(VTS_PGC1T_I) 102 is expressed by the number of 
bytes, relative to the first byte in the information table 
(VTS_PGCIT) 100. 

Furthermore, as shown in FIG. 16, the VTSLPGCIT 
search pointer (VTS_PGCIT_SRP) 103 contains the 
attributes (VTS_PGC_CAT) of the program chains 
(VTS_PGC) in the video title set (VTS) 72 and the start 
address (VTS_PGCLSA) of the VTS_PGC information 
(VTS_PGGI) 1 04 expressed by the number of bytes, rel- 
ative to the first byte in the VTS_PGC information table 
(VTS_PGC1T) 100. Here, the VTS_PGC attribute 
(VTS_PGG_CAT) is. for example, an attribute indicating 
whether an entry program chain (Entry PGC) is the first 
one to be reproduced. Usually, an entry program chain 
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(PGC) is written before program chains (PGC) that are 
not entry program chains (PGC). Generally, program 
chains (PGC) are reproduced in the description order of 
VTS_PGCIs as shown in FIG. 14. 

The PGC information (VTS_PGCI) 104 in the video 
title set 72 contains four tterrre as shown in FIG. 17. In 
the PGC information (VTS_PGCI) 104. the program 
chain general information (PGC_GI) 105 on a manda- 
tory item is first arranged, followed by at least three 
items that are made mandatory only when there is an 
video object 83. Specifically, contained as the three 
items in the PGC information (VTS_PGCI) 104 are a 
program chain program map (PGC_PGMAP) 106, a cell 
playback information table (C_PBIT) 107, and a cell 
position information table (C_POSiT) 108, 

As shown in FIG. 18, the program chain general 
information (PGC_GI) 105 contains the category 
(PGCI_CAT) of the program chain 87, the contents 
(PGC_CNT) of the program chain (PGC) 87. and the 
playback time of the program chain (PGC). 87 Written in 
the category of PGC (PGCLCAT) are whether the PGC 
can be copied or not and whether the programs 89 in 
the PGC are played back continuously or at random. 
The contents of PGC {PGC_CNT) contains the descrip- 
tion of the program chain structure, that is. the number 
of programs 89, the number of cells 84. and the number 
of angles in the program chain 87, The playback time of 
PGC (PGC_PB_TIME) contains the total playback time 
of the programs 89 in the PGC. The playback time is the 
time required to continuously play back the programs 89 
in the PGC, regardless of the playback procedure. 

Furthermore, the program chain general informa- 
tion (PGC_GI) 105 contains PGC sub-picture stream 
control (PGC_SPST_CTL), PGC audio stream control 
(PGC_AST_CTL). and PGC sub-picture pallet 
(PGC_SP_PLT). The PGC sub-picture stream control 
(PGC_SPST_CTL) contains the number of sub-pictures 
usable in the PGC 89 and the PGC audio stream control 
(PGC_AST_CTL) likewise contains the number of audio 
streams usable in the PGC 89. The PGC sub-picture 
palette (PGC__SP_PLT) contains a set of a specific 
number of color pallets used in all of the sub-picture 
streams in the PGC 89. 

Furthermore, the PGC general information 
(PGC_GI) 105 contains the start address (C_PBIT_SA) 
of the cell playback information table (C_PBIT) and the 
start address (C_POSIT_SA) of the cell position infor- 
mation table (C_POSIT) 103. Both of the start 
addresses (C_PBIT_SA and C_POSIT_SA) are repre- 
sented by the number of logical blocks, relative to the 
first byte in the VTS_PGC information (VTS_PGC1) 1 05. 

The program chain program map (PGC_PGMAP) 
108 is a map showing the arrangement of the programs 
89 in the PGC 87 of FIG. 19. In the map (PGC_PGMAP) 
106. the entry cell numbers (ECELLN). the start cell 
numbers of the individual programs 89, are written in 
ascending order as shown in FIGS. 19 arxJ 20. In addi- 
tion, program numbers are allocated, starting at 1 , in the 
order in which the entry cell numbers are written. Con- 



sequentiy, the first entry number in the map 
(PGC_PGMAP) 106 must be #1 . 

The cell playback information table (C_PBIT) 
defines the order in which tiie cells 84 in the PGC 87 are 

5 played back. In the cell playback information table 
(C_PB1T) 107, pieces of the cell playback information 
(C_PBI) are written consecutively as shown in FIG. 21. 
Basically, cells 84 are played back in the order of cell 
number. The cell playback information (C_PBIT) 108 

10 contains a cell category (C_CAT) as shown in FIG. 22. 
Written in the cell category (C_CAT) are a cell block 
mode indicating whetiier a cell is in the block and if it is 
in ttie block, whether the cell is the first one, a cell block 
type indicating whether a cell 84 is not part of the block 

15 or is in an angle block, and an STC discontinuity flag 
indicating whether the system time clock (STC) must be 
set again. 

Further written in the cell category (C_CAT) are a 
cell playback mode indicating whether the video object 

20 units (VOBU) 85 in the cell 84 are played t^ack continu- 
ously or a picture is made still at one video object unit 
(VOBU) 85 to another in the cell 84, and cell navigation 
control indicating whether the picture is made still after 
the playt>ack of the cell or indicating the rest time:. 

25 As shown in FIG. 22, the cell playback information 
table (C_PBIT) 107 contains the cell playback time 
(C_PBTNM) representing the total playback time of the 
PGC 87. When the PGC 87 has an angle cell block, tiie 
playback time of the angle cell number 1 represents the 

30 playback time of the angle block. Further written in the 
cell playback information table (C_PBIT) 107 are the 
start address (C_FVOBU_SA) of tiie first video object 
unit (VOBS) 85 in the cell 84 expressed by the number 
of logical sectors, relative to the first logical sector in tiie 

35 video object unit (VOBS) 85 in which the cell 84 is writ- 
ten and tiie start address (C_LVOBU_SA) of the end 
video object unit (VOBS) 85 in the cell 84 expressed by 
the number of logical sectors, relative to the first logical 
sector in the video object unit (VOBS) 85 in which the 

40 cell 84 is written. 

The cell position information table (C_POSI) 108 
specifies the identification number (VOB_ID) of the 
video object (VOB) 85 in the cell used in the PGC 87 
and the identification number (CellJD) of the cell 84. In 

45 tiie cell position information table (C_POSI) 1 08, pieces 
of the cell position information (C_POSI) corresponding 
to the cell numbers written in the cell playback informa- 
tion table 107 as shown in FIG. 23 are written in tiie 
same order as in the cell playback information table 

50 (C_PBIT) 107. The cell position information (C_POSI) 
contains the identification number (C_VOB_IDN) of the 
video object unit (VOBS) 85 in the cell 84 and tiie cell 
identification number (CJDN) as shown in FIG. 24. 
As explained with reference to FIG. 6, a cell 84 is a 

55 set of video object units (VOBU) 85. A video object unit 
85 is defined as a pack train starting with a navigation 
(NAV) pack 86. Therefore, the start address 
(C_FVOBU_SA) of tine first video object unit (VOBU) 85 
in a cell 84 is tiie start address of the NAV pack 86. As 
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shown in FIG. 25. the NAV pack 86 consists of a pack 
header 1 1 0, a system header 111, and two packets 1 1 6, 
117 of navigation data; namely, a presentation control 
information (PCi) packet 116 and a data search infor- 
mation (DSI) packet 117. As many bytes as shown in 5 
FIG. 25 are allocated to the respective sections so that 
one pack may contain 2048 bytes corresponding to one 
logical sector. The NV pack is provided immediately 
before the video pack 88 which contains the first data 
item in the group of pictures (GOP). Even if the object w 
unit 85 contains no video pack 88, the NV pack 86 is 
positioned at the head of the object unit 85, provided 
that the object unit 85 contains an audio pack 91 and/or 
a sub-picture pack 90. Thus, even if object unit 85 con- 
tains no video pacK rts playback time is determined on /5 
the basis of the time required for playing back a video 
pack 88, exactly in the same way as in the case where 
the unit 85 contains a video pack 88. 

Here, GOP is defined as a data train constituting a 
screen determined in the MPEG standards. In other 20 
words. GOP is equivalent to compressed data which 
may be expanded into image data representing a plural- 
ity of frames of a moving picture. The pack header 110 
contains a pack starting code, a system dock reference 
(SCR), and a multiplex rate. The system header 111 25 
contains a bit rate and a stream ID. The packet header 
1 1 2. 1 14 of each of the PCI packet 1 1 6 and DCI packet 
117 contains a packet starting code, a packet length, 
and a stream ID as determined in the MPEG2. 

As shown in FIG. 26, another video park 88. audio 30 
park 91 or sub-picture pack 90 consists of a pack 
header 120, packet header 121 , and a packet 122 con- 
taining the coH'esponding data as in the system layer 24 
MPEG 2. Its pack length is determined to be 2048 
bytes. Each of these packs is aligned with the bounda- 35 
ties between logical blocks. 

It is to be noted that PTS (Presentation Time 
Stamp; time management information for the repro- 
duced output) and DTS (Decoding Time Stamp; decod- 
ing time management information) are additionally 40 
stored in the packet header 1 21 of the video pack 88 
which includes head data of the I picture of GOP. PTS Is 
additionally stored in the packet header 1 21 of the audio 
pack 91 . PTS is additionally stored in the packet header 
121 of the sub-picture pack 90 which includes the head 45 
data of a sub-picture unit consisting of a plurality of sub- 
picture packs 90 for one still picture to be reproduced at 
the same time. 

SCR to be stored in each pack increases in the 
recording order on the optical disk 10 with the value of so 
the head pack of (video) data for each video object 
taken as 0. 

The PCI data (PCI) 113 in the PCI packet 116 Is 
navigation data used to make a presentation, or to 
change the contents of the display, in synchronization 55 
with the playback of the video data in the VOB unit 
(VOBU) 85. Specifically, the PCI data (PCI) 113 con- 
tains PCI general information (PCLGI) as information 
on the entire PCI data 1 13 change as shown in FIG. 27. 



The PCI general information (PCI_GI) contains the 
address (NV_PCK_LBN) of the NV pack (NV_PCK) 86 
in which the PCI data is recorded as shown in FIG. 28, 
the address being expressed in the number of blocks, 
relative to the VOBU logical sector in which the PCI data 
is recorded. The PCI general information (PCI_GI) con- 
tains tine category of VOBU (VOBU_CAT), the start PTS 
of VOBU 85 (VOBU_SPTS), and the end PTS of VOBU 
85 (VOBU^EPTS). Here, the start PTS of VOBU 85 
(VOBU_SPTS) indicates the playback start time (start 
presentation time stamp (SPTS)) of tiie video data In 
the VOBU 85 containing the PCI data. The playback 
start time is the playback start time of the first picture In 
the first GOP in the VOBU 85. Normally, the first picture 
corresponds to I picture (intra-picture) data In the MPEG 
standards. The end PTS (VOBU_EPTS) in the VOBU 
85 indicates the playback end time (end presentation 
time stamp (EPTS)) of the video data in the VOBU 85 
containing the PCI data. The playback end time is tiie 
playback start time of the last picture in the last GOP in 
the VOBU 85. 

The DSI data (DSI) 115 in the DSI packet 117 
shown in FIG. 25 is the navigation data used to search 
for a VOB unit (VOBU) 85. The DSI data (DSI) 1 15 con- 
tains the DSI general information (DSI_GI). VOBU 
search Information (VOBU_SI). and the synchronizing 
playback information (SYNCI) as shown in FIG. 29. 

The DSI information (DSI_GI) contains information 
about the entire DSI data 1 15. Specifically, as shown in 
FIG. 30, tiie DSI general information (DSLGI) contains 
the system clock reference for the NV pack 
(NV_PCK_SCR) 86. The system clock reference 
(NV_PCK_SCR) Is stored in tiie system time clock 
(STC) 54A. 58A. 60A or 62A built in each section of FIG. 
1 . On the basis of the STC 54A, 58A, 60A. 62A. video, 
audio, and sub-picture packs are decoded at the video, 
audio, and sub-picture decoders 58, 60, and 62 and tiie 
monitor 6 and the speaker 8 reproduce images and 
sound, respectively. The DSI general information 
(DSI_GI) contains the start address (NV_PCK_LBN) of 
the NV pack (NV_PCK) 86 containing the DSI data 
expressed by the number of logical sectors (RLSN), rel- 
ative to tiie first logical sector in the VOB set (VOBS) 82 
containing the DSI, and the address (VOBU_EA) of the 
last pack in the VOB unit (VOBU) 85 containing the DSI 
data expressed by the number of logical sectors 
(RLSN), relative to the first logical sector in the VOB unit 
(VOBU) 85. 

Furthermore, tiie DSI general information (DSI_GI) 
contains the end address (VOBU_IP_EA) of the V pack 
(V_PCK) 88 containing the last data item for the first I 
picture in the VOB unit (VOBU) expressed by the 
number of logical sectors (RLSN), relative to the first 
logical sector in the VOB unit (VOBU) 85 containing the 
DSI data, and the identification number (VOBU_IP_EA) 
of the VOB 83 containing the DSI and the identification 
number (VOBU_CJDN) of the cell 84 in which tiie DSI 
data is recorded. 
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The search information (VOBU_SI) in VOBU 85 
contains information used to identify the first address in 
a cell 84. 

The synchronizing information (SYNCI) contains 
address information on the sub-pictures and audio data 
reproduced in synchronization with the playback start 
time of the video data in the VOB unit (VOBU) 85 con- 
taining the DSI data. Specifically, as shown in FIG. 31 , 
the start address (A_SYNCA) of the target audio pack 
(A_PCK) is expressed by the number of logical sectors 
(RLSN). relative to the NV pack (NV_PCK) 86 in which 
DSI data 115 is recorded. When there are more than 
one audio stream (8 audio streams maximum), as many 
pieces of the synchronizing information (SYNCI) as 
there are audio streams are written. Furthermore, the 
synchronizing information (SYNCI) contains the 
address (SP„SYNCA) of the NV pack (NV_PCK) 86 of 
the VOB unit (VOBU) 85 containing the target audio 
pack {SP_PCK) 91 . the address being expressed by the 
number of logical sectors (RLSN). relative to the NV 
pack {NV_PCK) 86 in which DSI 1 15 is recorded. When 
there are more than one sub-picture stream (32 sub-pic- 
ture streams maximum), as many pieces of the syn- 
chronizing information (SYNCI) as there are sub-picture 
streams are written. 

Referring to FIG. 32, a sub-picture unit constituted 
of sub-picture data of a plurality of sub-picture packets 
will now be describe. A sub-picture unit as several doz- 
ens of still picture data (e.g.. superimposition) can be 
recorded in one GOP. The sub-picture unit consists of a 
sub-picture unit header (SPUH), pixel data (PXD) com- 
prised of run-length data, and a display control 
sequence table (DCSQT). 

Described in the sub-picture unit header (SPUH) 
are the size (SPDSZ) of the sub-picture unit and the 
start address (SPDCSQTA) of the display control 
sequence table, as shown in FIG. 33. 

Display control sequences (DCSQ) are described 
in a chronological order in the display control sequence 
table (DCSQT) as shown in FIG. 34. 

The start address (SPNDCSQA) and display con- 
trol command (SPDCCMD) of the next display control 
sequence are described in each display control 
sequence (DCSQ) as shown in FIG. 35. 

The display control command (SPDCCMD) con- 
sists of a command (FSTA_DSP) for setting the timing 
for starting the forced<li splay of pixel data, a command 
(STA_DSP) for setting the display start timing for pixel 
data, a command (STP_DSP) for setting the display end 
timing for pixel data, a command (SET_COLOR) for set- 
ting a color code of pixel data, a command 
(SET_CONTR) for setting the contrast between pixel 
data and main picture, a command (SET_DAREA) for 
setting the display area for pixel data, a command 
{SET_DSPXA) for setting the display start address for 
pixel data, a command (CHG_COLCON) for setting a 
color change and contrast change in pixel data, and a 
command (CMD_END) for terminating the display con- 
trol command. 



The command (STA_DSP) designates the display 
start time for sub-picture data, and is described by an 
offset PTS from PTS described in a sub-picture packet 
which includes a sub-picture unit header (start PTS). 
5 The command (STP_DSP) designates the display stop 
time for sub-picture data, and is described by the offset 
PTS from PTS descrtoed in a sub-picture packet which 
includes a sut)-picture unit header (stop PTS). 

The start PTS and stop PTS are delayed by arbi- 
10 trary predetermined times from those of main picture 
data and audio data which are reproduced in the same 
time slot. 

The system processor section 54 shown in FIG. 1 
has a packet discrimination/transfer section 54b, which 

15 compares the time in an internal STC timer 54a with the 
SCR (System Clock Reference) described in the pack 
header of each pack to discriminate the packet type of a 
matching pack, and sends the data of that packet to 
individual associated sections. The packet data of a 

20 navigation pack, which has been discriminated by this 
packet discrimination/transfer section 54b, is transferred 
to the data RAM section 56, the packet data of a video 
pack, discriminated by the section 54b, is transferred 
together with an input clock to the video buffer section 

25 59, the packet data of an audio pack, discriminated by 
the section 54b, is transferred togetiier with an input 
clock to the audio buffer section 61 . and the packet data 
of a sub-picture pack, discriminated by the section 54b. 
is transferred together with an input clock to the sub-pic- 

30 ture buffer section 63. 

As shown in FIGS. 36A to 36F, for example, when 
data is output packet by packet the input clock is output 
to the buffer associated with that packet type. 

Accordingly, the system processor section 54 

35 sequentially outputs sequences of video data, audio 
data and sub-picture data, which are to be reproduced 
in the same time slot, together with the input clock to the 
associated buffers. 

The system processor section 54 sends the PTS 

40 (Presentation Time Stamp) and DTS (Decoding Time 
Stamp), described in the packet header of a video data 
pack, the PTS, described in the packet header of an 
audio data pack, the start PTS and stop PTS, which are 
specified by the display control command (DCSQ) cind 

45 are included in the PTS, described in the packet header 
of a sub-picture data pack, and the time in the intemal 
STC timer 54a or the SCR to the buffer control/STC 
control section 57. 

The buffer control/STC control section 57 is 

so designed to store the SCR from the system processor 
section 54 into an STC timer 57a. outputs the SCR and 
the PTS and DTS of each data to the associated decod- 
ers, and outputs an output clock to the buffers associ- 
ated witii the PTS and DTS. 

55 For example, one unit of output clocks as shown in 
FIG. 36H is output to the video buffer section 59 in 
accordance with the DTS for video data as shown in 
FIG. 36G, one unit of output clocks as shown in FIG. 
36J is output to the audio buffer section 61 in accord- 
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ance with the PTS for audio data as shown in FIG. 361, 
and one unit of output clocks as shown in FIG. 36L is 
output to the sub-picture buffer section 63 in accord- 
ance with the start PTS for sub-picture data as shown in 
FIG. 36K. 5 

Each of the video buffer section 59, audio buffer 
section 61 and sub-picture buffer section 63 shown in 
FIG. 1 is an FIFO (First In First Out) memory which is a 
ring buffer comprised of FF circuits, as shown in FIG. 

37. If the capacity of this FIFO memory is set to approx- io 
Imately 10 times the VOBU (Video Object) recorded on 

the optical disk 10, data for several seconds to ten sec- 
onds can be retained. The FIFO memory stores data, 
supplied from the system processor section 54. in 
accordance with input data that is simultaneously sup- is 
plied, and outputs the stored data in response to the 
output clock supplied from the buffer control/STC con- 
trol section 57. 

The video decoder section 58 shown in FIG. 1 com- 
prises an STC timer 58a, registers 58b and 58c, compa- 20 
rators 58d and 58e. and a decoder 58f, as shown in FIG. 

38. The SCR set by the buffer control/STC control sec- 
tion 57 is stored in the STC timer 58a and the time 
therein is updated by a clock. The register 58b retains 

the DTS for video data which is set by the buffer con- 25 
trol/STC control section 57. The register 58c retains the 
PTS for video data which is set by the buffer con- 
trol/STC control section 57. The comparator 58d com- 
pares the STC in the STC timer 58a with the DTS in the 
register 58b to check if they match with each other. The 30 
comparator 58e compares the STC in the STC timer 
58a with the PTS in the register 58c to check if they 
match with each other. The decoder 58f decodes the 
video data input from the video buffer section 59 in 
accordance with the output of the comparator 58d and 35 
outputs the decoding result to the mixing section 64 in 
accordance with the output of the comparator 58d. 

The audio decoder section 60 shown in FIG. 1 com- 
prises an STC timer 60a, a register 6Qb, a comparator 
60c, and a decoder 60d, as shown in FIG. 39. The SCR 40 
set by the buffer control/STC control section 57 is stored 
in the STC timer 60a and the time therein is updated by 
a clock. The register 60a retains the PTS for audio data 
which is set by the buffer control/STC control section 57. 
The comparator 60c compares the STC in the STC 45 
timer 60a with the PTS in the register 60b to check if 
they match with each other. The decoder 60d decodes 
the audio data input from the audio buffer section 61 in 
accordance with the output of the comparator 58d when 
a match is made, and outputs the decoding result to the so 
audio D/A section 68. 

The sub-picture decoder section 62 shown in FIG. 1 
conrprises an STC timer 58a, registers 62b and 62c, 
conrparators 62d and 62e, and a decoder 62f, as shown 
in FIG. 40. The SCR set by the buffer control/STC con- ss 
trol section 57 is stored in the STC timer 62a and the 
time therein is updated by a clock. The register 62b 
retains the start PTS for sub-picture data which is set by 
the buffer control/STC control section 57. The register 



62c retains the stop PTS for sub-picture data which is 
set by the buffer control/STC control section 57. The 
comparator 62d compares the STC in the STC timer 
62a with the start PTS in the register 62b to check if they 
match with each other. The comparator 62e compares 
the STC in the STC timer 62a with the stop PTS in the 
register 62c to check if they match with each other. The 
decoder 62f starts decoding the sub-picture data input 
from the sub-picture buffer section 63 in accordance 
with the output of the comparator 62d when there is a 
match, and outputs the decoding result to the mixing 
section 64 in accordance with the output of the compa- 
rator 62d when there is a match. 

Referring again to FIG. 1 , a description will now be 
given of the operation of reproducing movie data on 
English conversations for learning English, from the 
optical disk 10 which has the logical format shown in 
FIGS, 4 to 31. 

When the optical disk apparatus shown in FIG. 1 is 
powered on and the optical disk 10 where movie data 
for English conversations for learning English is 
recorded is loaded, the system CPU section 50 reads 
the initialization program from the system ROM and 
RAM section 52. and activates the disk drive section 30. 
Therefore, the disk drive section 30 starts reading the 
lead-in area 27 and reads the volume and file structure 
area 70, which follows the lead-in area 27 and defines 
the volume and file structure in conform to ISO-9660 or 
the like. More specifically, to read the volume and file 
structure area 70 recorded at a predetermined position 
on the disk 10 set in the disk drive section 30, the sys- 
tem CPU section 50 sends a read command to the disk 
drive section 30 to read the contents of the volume and 
file structure area 70 and temporarily stores the con- 
tents in the data RAM section 56 via the system proces- 
sor section 54. The system CPU section 50 extracts 
information on the recorded position, recording capac- 
ity, size or the like of each file and other necessary man- 
agement information via the pass table and directory 
record stored in the data RAM section 56, transfers the 
extracted information at predetermined positions In the 
system ROM and RAM section 52 and saves the infor- 
mation there. 

Next, the system CPU section 50 refers to informa- 
tion on the recorded position and recording capacity of 
each file from the system ROM and RAM section 52 to 
acquire a video manager 71 , which consists of a plural- 
ity of files starting with a file number 0. That is, referring 
to the information on the recorded position and record- 
ing capacity of each file acquired from the system ROM 
and RAM section 52, the system CPU section 50 sends 
a read command to the disk drive section 30 to obtain 
the positions and sizes of a plurality of files which con- 
stitute the video manager 71 existing on the root direc- 
tory, reads this video manager 71 and stores the 
acquired information into the data RAM section 56 via 
the system processor section 54. A volume manager 
information management table (VMGI_MAT) 78, which 
is the first table in this video manager 71 . is searched. 
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Through this search. the start address 
(VMGM_VOBS_SA) of a video object set 
(VMGM_VOBS) 76 for a video manager menu (VMGM) 
is acquired and the video object set (VMGM_VOBS) 76 
is reproduced. As the procedures for reproducing this 
video object set (VMGM_VOBS) 76 for the video man- 
ager menu are the same as those for the video object 
set {VTSM_VOBS) for a title in the video title set (VTS). 
their description will not be repeated. When a language 
is set in the video object set (VMGM_VOBS) 76 or when 
there is no video manager menu (VMGM), the volume 
manager Information management table (VMGI_MAT) 
78 is searched to acquire the start address 
(TT_SRPT_SA) of a title search pointer table 
(TT_SRPT) 79. 

Through this search, the title search pointer table 
(TT_SRPT) 79 is transferred to a predetermined posi- 
tion In the system ROM and RAM section 52 to be 
saved there. Next, the system CPU section 50 acquires 
the end address of the title search pointer table 
(TT_SRPT) 79 from title search pointer table informa- 
tion (TSPTI) 92 and acquires the video title set number 
(VTSN)p program chain number (PGCN) and the start 
address (VTS_SA) of the video title set. which are asso- 
ciated with a number entered from the key/display sec- 
tion 4. from a title search pointer (TT_SRP) 93 
according to that input number. When there Is only one 
title set. one title search pointer (TT_SRP) 93 is 
searched to acquire the start address (VTS_SA) of that 
title set. irrespective of whether or not there is an input 
number from the key/display section 4. The system 
CPU section 50 acquires the target title set from the 
start address (VTS_SA) of this title set. 

The system CPU section 50 acquires the number of 
streams of the video, audio and sub-picture data for tiie 
volume menu and their attribute information, described 
in the volume manager information management table 
(VMGLMAT) 78 in a volume manager (VMGI) 75, and 
sets parameters for reproducing the video manager 
menu in the associated video decoder section 58, audio 
decoder section 60 arxj sub-picture decoder section 62 
based on the attribute information. 

Next, as shown in FIG. 12, video title set informa- 
tion (VTS!) 94 of the title set is acquired from the start 
address (VTS_SA) of a video title set 72. From a video 
title set information management table {VTSI_MAT) 98 
in this video title set information (VTSI) 94, the end 
address (VTI_MAT_EA) of the video title set information 
management table (VTSI_MAT) 98 shown in FIG. 13 is 
acquired. Based on the number of streams 
(VTS_AST_Ns, VTS_SPST_Ns) of audio data and sub- 
picture data and the attribute information (VTS_V_ATR, 
VTS_A_ATR, VTS_SPST_ATR) of the video, audio and 
sub-picture data, the incfividual sections of the repro- 
ducing apparatus shown in FIG. 1 are set according to 
their attributes. 

Cell playback information (C_PBI) shown in FIG. 21 
is acquired from a start address (C_POSIT_SA), and 
the start address (C_FVOBU_SA) of the first VOBU 85 



and the start addr^ (C_LVOBU_SA) of the last VOBU 
in the cell shown in FIG. 22. which are described in the 
playback information (C_PBI), are acquired to search 
for the target cell. The cell playback order is determined 

5 in such a way that playback cells 84 are determined one 
by one by referring to the program map shown in FIG. 
19 in a PGC program map (PGC_PGMAP) 106 shown 
in FIG. 17. The data cells 84 of the program chain, 
determined in this manner, are sequentially read from a 

10 video object 1 44 and are input to the data RAM section 
56 via the system processor section 54. The system 
processor section 54 sends each data cell 84 via the 
individual buffer sections 59. 61 and 63 to the video 
decoder section 58, audio decoder section 60 and sub- 

/5 picture decoder section 62 to be decoded based on the 
DTS as decode time information and the PTS as play- 
back time information. The video data and sub-picture 
data are mixed in the mixing section 64, after which the 
resultant data is converted to a video signal by the video 

20 D/A section 66 to reproduce an image on the monitor 
section 6. The audio data is converted to an audio sig- 
nal by the audio D/A section 68 which is sent to the 
speaker section 8 for sound reproduction. 

At this time, the superimposition as sub-picture. 

25 data is mixed with main picture data to be displayed 
when a predetermined time elapses after the display of 
the main picture data. 

The reproduction of video data will be described 
more specifically, with reference to the flowchart illus- 

30 trated in FIG. 41. 

In the reproduction of video data, when playback 
starts at step S1 1 as shown in FIG. 41 , the system CPU 
section 50 searches the volume manager (VMGI) 75 
and stores it in the system ROM artd RAM section 52 in 

35 Step SI 2 as has already been discussed above. Like- 
wise, the video title set information (VTSI) 94 of the 
video title set (VTS) 72 is read based on this volume 
manager (VMGI) 75, and the video title set menu is dis- 
played on the monitor section 6 using its video object 

40 set (VTSM_VOBS) 95. Based on this display, the title 
set 72 to be reproduced and the playback conditions or 
the like are determined by a user as indicated in step 
SI 3. When the determined title set 72 is selected using 
the key/display section 4, the system CPU section 50 

45 reads data of a cell playt^ack information table (C_PBIT) 
107. shown in FIGS. 17, 21 and 22, from a program 
chain information (VTS_PGCIT) 100 shown in FIG. 12 
in the selected title set 72 and stores the data in the sys- 
tem ROM and RAM section 52 as indicated in step SI 4. 

50 In accordance with the playback conditions entered 
from the key/display section 4, the system CPU section 
50 determines the program chain number 
(VTS_PGC_Ns) for starting playback, the audio stream 
number and sub-picture stream number, as indicated in 

55 step SI 5. For example, lessen 1 of the English conver- 
sation for learning English is selected as a title, and dis- 
playing superimposition as a sub-picture is determined. 
The determined sutj-picture number and audio stream 
number are set in the register (not shown) in the system 
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processor section 54 as indicated in step S16. Likewise, 
the playback start time is set in the system clock (STC) 
timers 54a,. 57a, 58a. 60a and 62a in the system proc- 
essor section 54, buffer control/STC control section 57, 
video decoder section 58 and audio decoder section 60 5 
and sub-picture decoder section 62. Further, the start 
address of the first VOBU in a cell as the start address 
and the PGC number or cell number are stored in the 
system ROM and RAM section 52. 

As indicated in step SI 7, the system CPU section 
50 sends a read command to the disk drive section 30 
when it is ready to read the video title set, and the disk 
drive section 30 seeks the optical disk 10 based on the 
aforementioned start address. In response to the read 
command, cells associated with the specified program 
chain (PGC) are sequentially read from the optical disk 
10, and are sent to the data RAM section 56 via the sys- 
tem CPU section 50 and system processor section 54. 
The transferred cell data is stored in the data RAM sec- 
tion 56 pack by pack, starting from navigation pack 86 20 
. as the head pack of the video object unit (VOBU) 85 as 
shown in FIG. 6. Thereafter, tiie system processor sec- 
tion 54 and buffer control/STC control section 57 distrib- 
ute the video data of the video pack 88 of the video 
object unit (VOBU) to the video decoder section 58 via 25 
the video buffer section 59, distribute the audio data of 
the audio pack 91 to tiie audio decoder section 60 via 
the audio buffer section 61 , and distribute the sub-pic- 
ture data of the sub-picture pack 90 to the sub-picture 
decoder section 62 via the sub-picture buffer section 63. 30 
The video data decoded by tiie video decoder section 
58 and the sub-picture data decoded by the sub-picture 
decoder section 62 are mixed by the mixing section 64. 
and the mixed data is sent to the video D/A section 66. 
The audio data decoded by tiie audio decoder section 35 
60 is sent to the audio D/A section 68. Consequentiy, a 
video signal is sent to the monitor section 6 and an 
audio signal is sent to the speaker section 8, so that the 
video display and audio playback will start. After a pre- 
determined period of time, the sub-picture is superim- 40 
posed on the display. 

When the playback is complete in step SI 8, or 
when the end PTS (VOBU_EPTS) described in general 
information (PCLGI) of the PCI 113 is referred to and 
this end PTS (VOBU_EPTS) matches with the system 45 
time clock (STC) as indicated in step SI 8, the screen 
display on the monitor section 6 is stopped as indicated 
in step SI 9. Then, the system CPU section 50 sends a 
data transfer stop command to the disk drive section 30 
to stop data transfer, terminating the playback operation so 
as indicated in step S20. 

This playback process will be described more spe- 
Cffically with reference to the tinning chart shown in 
FIGS. 36A through 36H and the flowchart illustrated in 
FIG. 42. 55 

As indicated in step S31 , the system processor sec- 
tion 54 sequentially reads tiie pack data, which is to be 
reproduced in the same time slot as tiie playt>ack time 
set in the STC timer 54a, from the data RAM section 56 



by referring to the SCR given to the pack header, deter- 
mines the packet type described in the packet header in 
that pack, and sends tiie packet data determined as tiie 
main picture to the video buffer section 59 as video 
data, the packet data determined as the audio data to 
the audio buffer section 61 as audio data, and the 
packet data determined as the sub-picture to the sub- 
picture buffer section 63 as sub-picture data in accord- 
ance with the determined packet type. 

As depicted in step S32, the system processor sec- 
tion 54 sends the time set in the STC timer 54a (SCR as 
the playback time), the DTS and PTS, described in tiie 
packet header of the head pack of the video data, tiie 
PTS, described in the packet header of the head pack of 
the audio data, and the start PTS and stop PTS, which 
are specified by the display control command (DCSQ) 
and are included in the PTS, described in the packet 
header of the head pack of the sub-picture data, to the 
buffer control/STC control section 57. 

The buffer control/STC control section 57 sets the 
supplied SCR as the playback time in the STC timer 58a 
of the video decoder section 58, the STC timer 60a of 
the audio decoder section 60 and STC timer 62a of the 
sub-picture decoder section 62 as specified in step S33. 

Further, the buffer control/STC control section 57 
outputs the input clock to the video buffer section 59 
when packet data of the video data is sent to the video 
buffer section 59 from the system processor section 54. 
outputs the input clock to the audio buffer section 61 
when the packet data of the audio data is sent to tiie 
audio buffer section 61 from the system processor sec- 
tion 54, and outputs the input clock to the sub-picture 
buffer section 63 when the packet data of the sub-pic- 
ture data is sent to the sub-picture buffer section 63 
from the system processor section 54. 

As a result, the video buffer section 59, audio buffer 
section 61 and sub-picture buffer section 63 store the 
supplied data in response to the input clock as 
described in step S34. 

When the STC in the STC timer 57a matches with 
the DTS of the video data, the buffer control/STC control 
section 57 outputs one unit of output clocks to the video 
buffer section 59. When tiie STC in the STC timer 58a 
in the video decoder section 58 matches with the DTS in 
the register 58b. the decoder 58f starts decoding based 
on the output of the comparator 58d. 

Consequently, the video data output from the video 
buffer section 59 in accordance with the output clock is 
decoded by the decoder 58f as indicated in step S35. 

As is apparent from step S36. when the STC in the 
STC timer 58a in the video decoder section 58 matches 
with the PTS in the register 58c, the decoded output of 
the decoder 58f is sent to the video D/A section 66 via 
the mixing section 64 based on the output from the com- 
parator 58e. 

When the STC in the STC timer 57a matches witii 
the PTS in the audio data, the buffer control/STC control 
section 57 outputs one unit of output clocks to the audio 
buffer section 61 . When the STC in the STC timer 60a 
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in the audio decoder section 60 matches with the PTS in 
the register 60b, the decoder 60d also starts decoding 
based on the output of the comparator 60c. 

Consequently, as depicted in step S37, the audio 
data output from the audio buffer section 61 in accord- 5 
ance with the output clock is decoded by the decoder 
60d and the decoded output is then sent to the audio 
D/A section 68. 

After a predetermined time has passed, i.e., when 
the STC in the STC timer 57a matches with the start io 
PTS of the sub-picture data, the buffer control/STC con- 
trol section 57 outputs one unit of output clocks to the 
sub-picture buffer section 63. When the STC in the STC 
timer 62a in the sub-picture decoder section 62 
matches with the start PTS in the register 62b, the is 
decoder 62f starts decoding based on the output of the 
comparator 62d. 

Consequently, as specified in step S38, the sub- 
picture data output from the sub-picture buffer section 
63 in accordance with the output clock is decoded by 20 
the decoder 62f and the decoded output is then mixed 
with the video data in the mixing section 64. The result- 
ant data is then sent to the video D/A section 66. 

When the STC in the STC timer 62a in the sub-pic- 
ture decoder section 62 matches with the stop PTS in 25 
the register 62c, the decoder 62f stops decoding based 
on the output of the comparator 62e. 

Accordingly, as indicated in step S39. the decoder 
62f in the sub-picture decoder section 62 stops output- 
ting the decoded data. 30 

Consequently, the image corresponding to the 
video data is displayed on the monitor section 6 and the 
voice corresponding to the audio data is reproduced 
from the speaker section 8. After a predetermined time 
delay, the mixed image corresponding to the sub-picture 35 
data and video data is displayed on the monitor section 
6. Thereafter, the sub-picture data will disappear. 

At the first timing, therefore, the image of a teacher 
pronouncing "ONE" is displayed as the main picture on 
the monitor section 6 and the teacher's voice "ONE" is 40 
reproduced from the speaker section 8. as shown in 
FIG. 43A. At the next timing, as shown in FIG. 43B, the 
image of the teacher saying "ONE" is displayed as the 
main picture on the monitor section 6 and the superim- 
position "ONE" as the sub-picture is superimposed on 45 
the main picture. At this timing, an apprentice as a lis- 
tener should pronounce the word. At the next timing, as 
shown in FIG. 43C, the image of the teacher saying 
"GOOD" is displayed as the main picture on the monitor 
section 6 and the teacher's voice "GOOD" is repro- so 
duced from the speaker section 8. 

As a result, the sub-picture which is to be displayed 
at the same time as the reproduction of the normal voice 
is delayed or the sub-picture is displayed with a some 
delay. In learning a language, for example, a listener ss 
can listen first without any hint and then confirm it later. 

The second embodiment of this invention will now 
be described, centering on the modified parts which dif- 



fer from the first embodiment, with reference to FIGS. 
43A to 43C and FIGS. 44 to 46. 

According to the second embodiment, the delay 
time Td for delaying sub-picture data is preset in the 
reproducing apparatus. 

In this embodiment, the start PTS described in the 
sub-picture unit as sub-picture data on the optical disk 
10 is the same as the PTS*s of video data and audio 
data which are to be processed in the same time slot. 

FIG. 44 presents a block diagram of an optical disk 
reproducing apparatus for explaining the second 
embodiment. 

The delay time Td for delaying sub-picture data is 
previously stored In the system ROM and RAM section 
52 or the internal memory (not shown) of the system 
CPU section 50. Accordingly, the delay time Td read 
from the system ROM and RAM section 52 by the sys- 
tem CPU section 50 is supplied to the buffer con- 
trol/STC control section 57. 

FIGS. 45A through 45M show a timing chart for 
explaining the relationship among an input clock, output 
clock. PTS and DTS for use in the reproduction of video 
data, audio data and sub-picture data. FIGS. 45A-45J 
are identical to FIGS. 36A through 36J. 

The buffer control/STC control section 57 sends the 
start PTS and stop PTS with respect to the sub-picture 
data, supplied from the system processor section 54. to 
the sub-picture decoder section 62 with the delay time 
Td, and also delays one unit of output clocks to be sent 
to the sub-picture buffer section 63 by the delay time Td. 

The start PTS and stop PTS for the sub-picture 
data supplied from the system processor section 54 are 
shown in FIG. 45K, the start PTS and stop PTS for the 
sub-picture data, which are to be delayed by the delay 
time Td, are shown in FIG. 45L and one unit of output 
clocks for the sub-picture data, which are to be delayed 
by the delay time Td. are shown in FIG. 45M. 

The playback operation of this structure will now be 
discussed referring to the timing chart shown in FIGS. 
45A-45M and the flowchart illustrated in FIG. 46. 

As Irxlicated in step S41 , the system processor sec- 
tion 54 sequentially reads the pack data, which is to be 
reproduced in the same time slot as the playback time 
set in the STC timer 54a, from the data RAM section 56 
by referring to the SCR given to the pack header, deter- 
mines the packet type described in the packet header in 
that pack, and sends the packet data determined as the 
main picture to the video buffer section 59 as video 
data, the packet data determined as the audio data to 
the audio buffer section 61 as audio data, and the 
packet data determined as the sub-picture to the sub- 
picture buffer section 63 as sub-picture data in accord- 
ance with the determined packet type. 

As depicted in step S42. the system processor sec- 
tion 54 sends the time set in the STC timer 54a (SCR as 
the playtack time), the DTS and PTS, described in tine 
packet header of the head pack of the video data, tiie 
PTS, described in the packet header of the head pack of 
the audio data, and the start PTS arxJ stop PTS. which 
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are specified by the display control command (DCSQ) 
and are included in the PTS, described in the packet 
header of the head pack of the sub-picture data, to the 
buffer control/STC control section 57. The system CPU 
section 50 supplies the delay time Td for the sub-picture 
data, read from the system ROM and RAM section 52. 
to this buffer control/STC control section 57. 

The buffer control/STC control section 57 sets the 
supplied SCR as the playback time in the STC timer 58a 
of the video decoder section 58, the STC timer 60a of 
the audio decoder section 60 and STC timer 62a of the 
sub-picture decoder section 62 as specified in step S43. 

Further, the buffer control/STC control section 57 
outputs the Input clock to the video buffer section 59 
when packet data of the video data is sent to the video 
buffer section 59 from the system processor section 54, 
outputs the input clock to the audio buffer section 61 
when the packet data of the audio data is sent to the 
audio buffer section 61 from the system processor sec- 
tion 54, and outputs the input clock to the sub-picture 
buffer section 63 when the packet data of the sub-pic- 
ture data is sent to the sub-picture buffer section 63 
from the system processor section 54. 

As a result, the video buffer section 59, audio buffer 
section 61 and sub-picture buffer section 63 store the 
supplied data in response to the input clock as indicated 
in step S44. 

When the STC in the STC timer 57a matches with 
the DTS of the video data, the buffer control/STC control 
section 57 outputs one unit of output clocks to the video 
buffer section 59. When the STC in the STC timer 58a 
in the video decoder section 58 matches with the DTS in 
the register 58b. the decoder 58f starts decoding based 
on the output of the comparator 58d. 

Consequently, the video data output from the video 
buffer section 59 in accordance with the output clock is 
decoded by the decoder 58f as described in step S45. 

As is apparent from step S46. when the STC in the 
STC timer 58a in the video decoder section 58 matches 
with the PTS in the register 58c. the decoded output of 
the decoder 58f is sent to the video D/A section 66 via 
the mixing section 64 based on the output from the com- 
parator 58e. 

When the STC in the STC timer 57a matches with 
the PTS in the audio data, the buffer control/STC control 
section 57 outputs one unit of output clocks to the audio 
buffer section 61, When the STC in the STC timer 60a 
in the audio decoder section 60 matches with the PTS in 
the register 60b, the decoder 60d also starts decoding 
based on the output of the comparator 60c. 

Consequently, as depicted in step S47, the aucfio 
data output from the audio buffer section 61 in accord- 
ance with the output clock is decoded by the decoder 
60d and the decoded output is then sent to the audio 
D/A section 68. 

After a predetermined time has passed, i.e.. when 
the delay time Td elapses after the matching of the STC 
in the STC timer 57a with the start PTS of the sub-pic- 
ture data, the buffer control/STC control section 57 out- 



puts one unit of output clocks to the sub-picture buffer 
section 63. When the STC in the STC timer 62a in the 
sub-picture decoder section 62 matches with the start 
PTS delayed by the delay time Td in the register 62b. 
5 the decoder 62f starts decoding based on the output of 
the comparator 62d. 

Consequently, as indicated in step S48, the sub- 
picture data output from the sub-picture buffer section 
63 in accordance with the output clock is decoded by 
10 the decoder 62f and the decoded output is tiien mixed 
with the video data in the mixing section 64. The result- 
ant data is then sent to the video D/A section 66. 

When the STC in the STC timer 62a in the sub-pic- 
ture decoder section 62 matches with the stop PTS 
15 delayed by the delay time Td in the register 62c, the 
decoder 62f stops decoding teased on the output of the 
comparator 62e. 

Accordingly, as specified in step S49, the decoder 
62f in the sub-picture decoder section 62 stops output- 
ting the decoded data. 

Consequently, the image corresponding to the 
video data is displayed on the monitor section 6 and the 
voice corresponding to the audio data is reproduced 
from the speaker section 8. After a predetermined time 
delay, the mixed image corresponding to the sub-picture 
data and video data is displayed on the monitor section 
6. Thereafter, the sub-picture data will disappear. 

At the first timing, therefore, the image of a teacher 
saying "ONE" is displayed as the main picture on the 
monitor section 6 and the teacher's voice "ONE" is 
reproduced from the speaker section 8, as shown in 
FIG. 43A. At the next timing, as shown in FIG. 43B, the 
image of the teacher pronouncing "ONE" is displayed as 
the main picture on the monitor section 6 and the super- 
imposition "ONE" as the sub-picture is superimposed 
on the main picture. At this timing, an apprentice as a 
listener should pronounce the word. At the next timing, 
as shown in FIG. 43C, the image of the teacher saying 
"GOOD" is displayed as the main picture on the monitor 
section 6 and tiie teacher's voice "GOOD" is repro- 
duced from the speaker section 8. 

The third embodiment of this invention will now be 
discussed with reference to FIGS. 43A through 43C, 
FIG. 44, FIGS. 45A tiirough 45M, and FIGS. 47A, 47B 
and 48. The delay time at which the subpicture data is 
reproduced with respect to the main picture data is pre- 
set in the reproducing apparatus in the second embodi- 
ment, whereas a user can freely set the delay time 
using the key/display section 4 and what is displayed on 
the monitor section 6 in the third embodiment. The 
key/display section 4 may be. constituted of a remote 
controller, for example. 

For instance, the basic menu screen for setting the 
delay time as shown in FIG. 47A is displayed on the 
monitor section 6, and a user selects "DELAY SET" for 
setting the delay time from among the selections 
"TITLE," "LANGUAGE" and "DELAY SET" on the basic 
menu screen using the key/display section 4. Then, the 
delay time setting screen as shown in FIG. 47B is dis- 
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played on the monitor section 6. Using the key/display 
section 4. the user selects, for example, "SUB-PIC- 
TURE" from "VIDEO," "AUDIO'* and "SUB-PICTURE" 
on the delay time setting screen, as the type for which 
the delay time is to be set. and then sets the delay time 5 
(seconds) through the time setting section on that 
screen. 

The playback operation of this structure will now be 
discussed referring to the timing chart shown in FIGS. 
45A-45M and the flowchart illustrated in FIG. 48. 10 

First, the basic menu screen for setting the delay 
time as shown in FIG. 47A is displayed on the monitor 
section 6 as Indicated in step S50. The user then 
selects "DEI_AY SET" for setting the delay time from 
among "TITLE," "LANGUAGE" and "DELAY SET" on is 
the basic menu screen using the key/display section 4 
as indicated in step S51. 

Consequently, the time setting screen for each type 
to be delayed (hereinafter referred to as "type-by-type 
delay time setting screen") as shown In FIG. 47B is dis- 20 
played on the monitor section 6 as indicated in step 
852. Then, using the key/display section 4. the user 
selects, for example. "SUB-PICTURE" from among 
"VIDEO." "AUDIO" and "SUB-PICTURE" on the type- 
by-type delay time setting screen, as the type for which 25 
the delay time is to be set, and then sets the delay time 
(seconds) through the time setting section on that 
screen as depicted in step S53. 

Then, as indicated in step S54. the system proces- 
sor section 54 sequentially reads the pack data, which 30 
is to be reproduced in the same time slot as the play- 
back time set in the STC timer 54a, from the data RAM 
section 56 by referring to the SCR given to the pack 
header, determines the packet type described in the 
packet header in that pack, and sends the packet data 35 
determined as the main picture to the video buffer sec- 
tion 59 as video data, the packet data determined as the 
audio data to the audio buffer section 61 as audio data, 
and the packet data determined as the sub-picture to 
the sub-picture buffer section 63 as sub-picture data in 40 
accordance with the determined packet type. 

As depicted in step S55, the system processor sec- 
tion 54 sends the time set in the STC timer 54a (SCR as 
the playback time), the DTS and PTS, described in the 
packet header of the head pack of the video data, the 45 
PTS, described in the packet header of the head pack of 
the audio data, and the start PTS and stop PTS. which 
are specified by the display control command (DCSQ) 
and are included in the PTS, described in the packet 
header of the head pack of the sut>picture data, to the so 
buffer control/STC control section 57. The system CPU 
section 50 supplies the type targeted for the delay, set 
by the key/display section 4, the delay time Td for the 
sub-picture data, read from the system ROM and RAM 
section 52. to this buffer control/STC control section 57. ss 

The buffer control/STC control section 57 sets the 
supplied SCR as the playback time in the STC timer 58a 
of the video decoder section 58, the STC timer 60a of 



the audio decoder section 60 and STC timer 62a of the 
sub-picture decoder section 62 as specified in step S56. 

Further, the buffer control/STC control section 57 
outputs the input clock to the video buffer section 59 
when packet data of the video data is sent to the video 
buffer section 59 from the system processor section 54, 
outputs the input clock to the audio buffer section 61 
when the packet data of the audio data is sent to the 
audio buffer section 61 from the system processor sec- 
tion 54. and outputs the input clock to the sub-picture 
buffer section 63 when the packet data of the sub-pic- 
ture data is sent to the sub-picture buffer section 63 
from the system processor section 54. 

As a result, the video buffer section 59. audio buffer 
section 61 and sub-picture buffer section 63 store the 
supplied data in response to the input clock as indicated 
in st^ S57. 

When the STC in the STC timer 57a matches with 
the DTS of the video data, the buffer control/STC control 
section 57 outputs one unit of output clocks to the video 
buffer section 59. When the STC in the STC timer 58a 
in the video decoder section 58 matches with the DTS in 
the register 58b, the decoder 58f starts decoding based 
on the output of the comparator 58d. 

Consequently, the video data output from the video 
buffer section 59 in accordance with the output clock is 
decoded by the decoder 58f as described in step S58. 

As is apparent from step S59. when the STC in the 
STC timer 58a in the video decoder section 58 matches 
with the PTS in the register 58c. the decoded output of 
the decoder 58f is sent to the video D/A section 66 via 
the mixing section 64 based on the output from the com- 
parator 58e. 

When the STC in the STC timer 57a matches with 
the PTS in the audio data, the buffer control/STC control 
section 57 outputs one unit of output clocks to the audio 
buffer section 61 . When the STC in the STC timer 60a 
in the audio decoder section 60 matches with the PTS in 
the register 60b, the decoder 60d also starts decoding 
based on the output of the comparator 60c. 

Consequently, as depicted in step S60. the audio 
data output from the audio buffer section 61 in accord- 
ance with the output clock is decoded by the decoder 
60d and the decoded output is then sent to the audio 
D/A section 68. 

After a predetermined time has passed, i.e., when 
the delay time Td elapses after the matching of the STC 
in the STC timer 57a with the start PTS of the sub-pic- 
ture data, the buffer control/STC control section 57 out- 
puts one unit of output clocks to the sub-picture buffer 
section 63. When the STC in the STC timer 62a in the 
sub-picture decoder section 62 matches with the start 
PTS delayed by the delay time Td in the register 62b. 
the decoder 62f starts decoding based on the output of 
the comparator 62d. 

Consequently, as indicated in step S61. the sub- 
picture data output from the sub-picture buffer section 
63 in accordance with the output clock is decoded by 
the decoder 62f and the decoded output is then mixed 
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with the video data in the mixing section 64. The result- 
ant data is then sent to the video D/A section 66. 

When the STC in the STC timer 62a In the sub-pic- 
ture decoder section 62 matches with the stop PTS 
delayed by the delay time Td in the register 62c. the s 
decoder 62f stops decoding based on the output of the 
comparator 62 e. 

Accordingly, as specified in step 362. the decoder 
62f in the sub-picture decoder section 62 stops output- 
ting the decoded data. io 

Consequently, the image corresponding to the 
video data is displayed on the monitor section 6 and the 
voice corresponding to the audio data is reproduced 
from the speaker section 8. After a predetermined time 
delay, the mixed image corresponding to the sub-picture is 
data and video data is displayed on the monitor section 
6. Thereafter, the sub-picture data will disappear. 

At the first timing, therefore, the image of a teacher 
saying "ONE" is displayed as the main picture on the 
monitor section 6 and the teacher's voice "ONE" is 20 
reproduced from the speaker section 8, as shown in 
FIG. 43 A. At the next timing, as shown in FIG. 43B, the 
image of the teacher pronouncing "ONE" is displayed as 
the main picture on the monitor section 6 and the super- 
imposition "ONE" as the sub-picture is superimposed 25 
on the main picture. At this timing, an apprentice as a 
listener should pronounce the word. At the next timing, 
as shown in FIG. 430. the image of the teacher saying 
"GOOD" is displayed as the main picture on the monitor 
section 6 and the teacher's voice "GOOD" is repro- 30 
duced from the speaker section 8. 

Although the foregoing description of this embodi- 
ment has been given with reference to the case where 
sub-picture data is delayed, this invention may also be 
adapted to the case where main picture data or audio 35 
data is delayed. 

The fourth embodiment of this invention will now be 
described referring to FIG. 1, FIGS. 49A through 49N, 
FIG. 50 and FIGS, 51 A through 510. According to the 
fourth embodiment, by utilizing the fact that the display 40 
period of a sub-picture which is expressed by using the 
start PTS arxl stop PTS of a sub-picture unit corre- 
sponds to one phrase of the normal speech, the display 
timing of a sub-picture is delayed by one phrase and 
during the display of this sub-picture, the main picture 45 
and voice are stopped or frozen. 

In this embodiment, the start PTS described in the 
sub-picture unit as sub-picture data on the optical disk 
10 is the same as the PTS's of video data and audio 
data which are to be processed in the same time slot. so 

FIGS. 49A through 49M show a timing chart for 
explaining the relationship among an input clock, output 
clock, PTS and DTS for use in the reproduction of video 
data, audio data and sub-picture data. FIGS. 49A-49F 
are identical to FIGS. 36 A through 36F 55 

The buffer control/STC control section 57 delays 
the start PTS and stop PTS with respect to the sub-pic- 
ture data, supplied from the system processor section 
54. by the delay time Td corresponding to one phrase 



computed from the difference between the start PTS 
and stop PTS. and outputs the delayed start PTS and 
stop PTS to the sub-picture decoder section 62. and 
also delays one unit of output clocks to be sent to the 
sub-picture buffer section 63 by the delay time Td. 

The start PTS and stop PTS for the sub-picture 
data supplied from the system processor section 54 are 
shown in FIG. 49K. the start PTS and stop PTS for the 
sub-picture data, which are to be delayed by the delay 
time Td, are shown in FIG. 49L and one unit of output 
clocks for the sub-picture data, which are to be delayed 
by the delay time Td, are shown in FIG. 49M. 

During the period from the start PTS for the sub- 
picture data output with the delay time Td to the stop 
PTS, the buffer control/STC control section 57 outputs a 
stop signal to the STC timer 58a in the video decoder 
section 58 and the STC timer 60a in the audio decoder 
section 60. 

In other words, the buffer control/STC control sec- 
tion 57 stops the STC timers 58a and 60a in accordance 
with the start PTS delayed by the delay time Td, and 
sets the time leading by the delay time Td from the STC 
corresponding to the start PTS delayed by the delay 
time in the STC timers 58a and 60a in accordance with 
the stop PTS. 

While a sub-picture is being displayed, freezing the 
video and audio data is carried out. 

The playback operation of this structure will now be 
discussed referring to the timing chart shown in FIGS. 
49A-49N and the flowchart illustrated in FIG. 50. 

As indicated in step S71 , the system processor sec- 
tion 54 sequentially reads the pack data, which is to be 
reproduced in the same time slot as the playback time 
set in the STC timer 54a, from the data RAM section 56 
by referring to the SCR given to the pack header, deter- 
mines the packet type described in the packet header in 
that pack, and sends the packet data determined as the 
main picture to the video buffer section 59 as video 
data, the packet data determined as the audio data to 
the audio buffer section 61 as audio data, and the 
packet data determined as the sub-picture to the sub- 
picture buffer section 63 as sub-picture data in accord- 
ance with the determined packet type. 

As indicated in step S72, the system processor sec- 
tion 54 sends the time set in the STC timer 54a (SCR as 
the playback time), the DTS and PTS, described in the 
packet header of the head pack of the video data, the 
PTS. described in the packet header of the head pack of 
the audio data, and the start PTS and stop PTS. which 
are specified by the display control command (DCSQ) 
and are included in the PTS, described in the packet 
header of the head pack of the sub-picture data, to the 
buffer control/STC control section 57. 

The buffer control/STC control section 57 sets the 
supplied SCR as the playback time in the STC timer 58a 
of the video decoder section 58, the STC timer 60a of 
the audio decoder section 60 and STC timer 62a of the 
sub-picture decoder section 62 as indicated in step S73. 
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The buffer controiySTC control section 57 computes 
the delay time Td corresponding to one phrase from the 
difference between the start PTS and stop PTS for the 
subiDicture data. 

Further, the buffer control/STC control section 57 s 
outputs the Input dock to the video buffer section 59 
when packet data of the video data is sent to the video 
buffer section 59 from the system processor section 54. 
outputs the input clock to the audio buffer section 61 
when the packet data of the audio data is sent to the io 
audio buffer section 61 from the system processor sec- 
tion 54, and outputs the input dock to the sub-picture 
buffer section 63 when the packet data of the sub-pic- 
ture data is sent to the sub-picture buffer section 63 
from the system processor section 54. is 

As a result, the video buffer section 59, audio buffer 
section 61 and sub-picture buffer section 63 store the 
supplied data in response to the input dock as indicated 
in step S74. 

When the STC in the STC timer 57a matches with 20 
the DTS of the video data, the buffer control/STC control 
section 57 outputs one unit of output clod<s to the video 
buffer section 59. When the STC in the STC timer 58a 
in the video decoder section 58 matches with the DTS in 
the register 58b. the decoder 58f starts decoding based 25 
on the output of the comparator 58d. 

Consequently, the video data output from the video 
buffer section 59 in accordance with the output dock is 
decoded by the decoder 58f as indicated in step S75. 

As is apparent from step S76, when the STC in the 30 
STC timer 58a in the video decoder section 58 matches 
with the PTS in the register 58c, the decoded output of 
the decoder 58f is sent to the video D/A section 66 via 
the mixing section 64 based on the output from the com- 
parator 58e. 35 

When the STC in the STC timer 57a matches with 
the PTS in the audio data, the buffer control/STC control 
section 57 outputs one unit of output docks to the audio 
buffer section 61. When the STC in the STC timer 60a 
in the audio decoder section 60 matches with the PTS in 40 
the register 60b, the decoder 60d also starts decoding 
based on the output of the comparator 60c. 

Consequently, as indicated in step S77. the audio 
data output from the audio buffer sedion 61 in accord- 
ance with the output clock is decoded by the decoder 45 
60d and the decoded output is then sent to the audio 
D/A section 68. 

After a predetermined time has passed, i.e., when 
the computed delay time Td elapses after the matching 
of the STC in the STC timer 57a with the start PTS of so 
the suk>-picture data, the buffer control/STC control sec- 
tion 57 outputs one unit of output clocks to the sub-pic- 
ture buffer section 63. When the STC in the STC timer 
62a in the sub-picture decoder section 62 matches with 
the start PTS delayed by the delay time Td in the regis- ss 
ter 62b, the decoder 62f starts decoding based on the 
output of the comparator 62d. 

Consequently, as indicated in step S78, the sub- 
picture data output from the sub-picture buffer section 



63 in accordance with the output clock is decoded by 
the decoder 62f and the decoded output is then mixed 
with the video data in the mixing section 64. The result- 
ant data is then sent to the video D/A section 66. 

At this time, the buffer control/STC control section 
57 supplies a stop signal to the STC timer 58a in the 
video decoder section 58 and the STC timer 60a in the 
audio decoder section 60. Consequently, decoding by 
the decoder 5df and decoding by the decoder 60d stop 
artd the video image on the screen when the decoding 
operations are stopped is displayed on the monitor sec- 
tion 6 with no sounds generated. 

When the STC in the STC timer 62a in the sub-pic- 
ture decoder section 62 matches with the stop PTS 
delayed by the delay time Td in the register 62c, the 
decoder 62f stops decoding based on the output of the 
comparator 62e. 

At this time, the buffer control/STC control section 
57 sets the time leading by the delay time Td from the 
STC corresponding to the start PTS, delayed by the 
delay time Td, in the STC timer 58a in the video decoder 
section 58 and the STC timer 60a in the audio decoder 
section 60. As a result, the decoders 58f and 60d restart 
decoding. 

Accordingly, as indicated in step S79. the decoder 
62f in the sub-picture decoder section 62 stops output- 
ting the decoded data, and the decoded output of the 
decoder 58f is supplied to the video D/A section 66 via 
the mixing section 64 while the decoded output of the 
decoder 60d is supplied to the audio D/A section 68. 

Consequently, the image corresponding to the 
video data is displayed on the monitor section 6 and the 
voice corresponding to the audio data is reproduced 
from the speaker section 8. After a predetermined time 
delay corresponding to one phrase, the mixed image 
corresponding to the sub-picture data and video data is 
displayed on the monitor section 6. At this time, the 
video and audio data are frozen. Thereafter, the sub- 
picture data will disappear and the video and audio data 
are reproduced from that time. 

At the first timing, therefore, the image of a teacher 
saying "ONE" is displayed as the main picture on the 
monitor section 6 and the teacher's voice "ONE" is 
reproduced from the speaker section 8, as shown in 
FIG. 51 At the next timing, as shown in FIG. 51 B. the 
image of the teacher having said "ONE" and the super- 
imposition "ONE** as the sub-picture are displayed in the 
superimposing manner on the monitor section 6. At this 
timing, an apprentice as a listener should pronounce the 
word. At the next timing, as shown in FIG. 51 C, the 
image of the teacher saying "GOOD" is displayed as the 
main picture on the monitor section 6 and the teacher's 
voice "GOOD" is reproduced from the speaker section 
8. 

As a result, at the time of learning a language, a 
student can make a pronunciation while referring to the 
superimposition after the pronundation by the teacher. 

As shown in FIGS. 52A-52C. the superimposition 
as a sub-picture may be displayed previously, the main 
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picture and sound may be frozen for the display period 
of this sub-picture, and the main picture and sound may 
be reproduced after the display of the sub-picture is 
completed. 

First, at the first timing, the picture of the teacher is s 
displayed on the monitor section 6 as the main picture 
as shown in FIG. 52A. At the next timing, the image of 
the teacher stilt on the screen as the main picture and 
the superimposition "TWO" as the sub-picture are 
superimposed on the monitor section 6 as shown in io 
FIG. 52B. At this timing, an apprentice as a listener is 
expected to make a pronunciation. At the next timing, 
the image of the teacher saying "TWO" is displayed on 
the monitor section 6 and the teacher's voice "TWO" is 
reproduced from the speaker section 8, as shown in 75 
FIG. 52G. 

tn this case, the delay times for the main picture and 
sound may be set from the menu and delaying may be 
executed accordingly, or delaying may be done by the 
time specified by the start PTS and stop PTS for the 20 
sub-picture unit. 

As shown in FIGS. 53A-53C, the sound for only the 
phrase portion may be delayed and then reproduced, 
the main picture and the superimposition as the sub- 
picture may be frozen for the playback period of this 25 
sound, and the main picture and sound may be repro- 
duced after the sound reproduction is completed. 

First, at the first timing, the picture of the teacher as 
the main picture and the superimposition "THREE" as 
the sub-picture are superimposed on the monitor sec- 30 
Won 6 as shown in FIG. 53A. At this timing, the appren- 
tice as a listener is expected to make a pronunciation. At 
the next timing, the image of the teacher pronouncing 
"THREE" is displayed on the monitor section 6 and the 
teacher's voice "THREE" is reproduced from the 35 
speaker section 8, as shown in FIG. 53B. At tiie next 
timing, as shown in FIG. 54C, the image of the teacher 
saying "GOOD" as the main picture is displayed on the 
monitor section 6 and the teacher's voice "GOOD" is 
reproduced from the speaker section 8. 40 

In this case, the delay times for the main picture and 
sound may be set from the menu and delaying may be 
executed accordingly, or delaying may be done by the 
time specified by the start PTS and stop PTS for the 
sub-picture unit, as the phrase period. 45 

As shown in FIGS. 54A-54C, when the first sound 
and second sound are present in addition to the main 
picture and sub-picture, the second sound may be 
reproduced with a delay, the main picture, the superim- 
position as the sub-picture and the first sound may be so 
frozen for the playback period of the second sound, and 
the main picture, the sub-picture and the first sound 
may be reproduced after the reproduction of the second 
sound is conrpleted. 

At the first timing, the image of the teacher saying 55 
"FOUR" as the main picture and the superimposition 
"FOUR" as the sub-picture are superimposed on the 
monitor section 6. and the first voice "FOUR" by the 
teacher is reproduced from the speaker section 8, as 
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shown in FIG. 54A. At the next timing, as shown in FIG. 
54B, the image of the teaching having said "FOUR" as 
the main picture is kept displayed on the monitor section 
6. and the second voice "YON" (in Japanese) by the 
teacher is reproduced from the speaker section 8. At 
this timing, an apprentice as a listener can know the 
translation of the English word. At the next timing, as 
shown in FIG. 54C, the image of the teacher saying 
"GOOD" as the main picture is displayed on the monitor 
section 6 and the teacher's first voice "GOOD" is repro- 
duced from tile speaker section 8. 

As a result, the student can listen to the translation 
of the conversation at the delayed timing at the time of 
learning the language. 

In this case, the delay times of the main picture, 
sub-picture and the first sound may be set from the 
menu, or delaying may be done by the time specified by 
the start PTS and stop PTS for the sub-picture unit. 

As shown in FIGS. 55A-55C, when the first sound 
and second sound are present in addition to the main 
picture and sub-picture, the first sound and the superim- 
position as the sub-picture may be reproduced with a 
certain delay, the main picture and the second sound 
may be frozen for the playback period of the first sound 
and the sub-picture, and the main picture and the sec- 
ond sound may be reproduced after the reproduction of 
the first sound and sub-picture is completed. 

At the first timing, as shown in FIG. 55A, the image 
of the teacher saying "GO" (in Japanese) as the main 
picture is displayed on the monitor section 6, and the 
second voice "GO' by tiie teacher is reproduced from 
the speaker section 8. At the next timing, as shown in 
FIG. 55B, the image of the teaching having said "GO" as 
the main picture and the superimposition "FIVE" as the 
sub-picture are superimposed on the monitor section 6, 
and the first voice "FIVE" by the teacher is reproduced 
from the speaker section 8. At this timing, an apprentice 
as a listener can know the translation of the English 
word. At the next timing, as shown in FIG. 55C, the 
image of the teacher saying "GOOD" as the main pic- 
ture is displayed on the monitor section 6 and the 
teacher's first voice "GOOD" is reproduced from the 
speaker section 8. 

As a result, the student can listen to the translation 
of the conversation at the delayed timing at the time of 
learning the language. 

In this case, the delay times of the first sound and 
the sub-picture may be set from the menu, or delaying 
may be done by the time specified by the start PTS and 
stop PTS for the sub-picture unit. 

As shown in FIGS. 56A-56C, when the first sub-pic- 
ture and second sub-picture are present in addition to 
the main picture and sound, the second sub-picture 
may be reproduced with a certain delay, the main pic- 
ture and sound may be frozen for the playback period of 
the second sub-picture, and the main picture and sound 
may be reproduced after the reproduction of the second 
sub-prcture is completed. 
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At the first timing, as shown in FIG. 56A, the image 
of the teacher saying "SIX" as the main picture and the 
superimposition "SIX" as the first sub-picture are super- 
imposed on the monitor section 6. and the teacher's 
voice "SIX" is reproduced from the speaker section 8. At 5 
the next timing, as shown in FIG. 56B. the image of the 
teaching having said "SIX" as the main picture and the 
superimposition "ROKU" (in Japanese) as the second 
sub-picture or translation are superimposed on the 
monitor section 6. At this timing, an apprentice as a lis- ro 
tener can know the translation of the English word. At 
the next timing, as shown in FIG. 56C. the image of the 
teacher saying "GOOD" as the main picture is displayed 
on the monitor section 6 and the teacher's voice 
"GOOD" is reproduced from the speaker section 8. rs 

As a result, the student can listen to the translation 
of the conversation at the delayed timing at the time of 
learning the language. 

In this case, the delay time of the second sub-pic- 
ture may be set from the menu, or delaying may be 20 
done by the time specified by the start PTS and stop 
PTS for the sub-picture unit of the second sub-picture. 

Alternatively, the first and second sub-pictures may 
be displayed simultaneously, in which case separate 
display positions should be set for the first and second 25 
sub-pictures or the display colors should be changed for 
the distinction. 

Instead of delaying sub-picture data, the sub-pic- 
ture data may be displayed as if hidden by a mosaic or 
black filling for the delay time. so 

In this case, as shown in FIG. 57, a digital signal 
processor (DSP) 65 which selectively turns the sub-pic- 
ture data from the sub-picture decoder section 62 into a 
mosaic pattern or painting the data with black is pro- 
vided between the sub-picture decoder section 62 and 35 
the mixing section 64 in the structure in FIG. 1 . 

In the case where an 8 x 8 dot mosaic is to be gen- 
erated, this DSP 65 comprises a first sample and hold 
section for sampling and holding data input dot by dot, a 
first line memory for storing the output of the first sam- 40 
pie and hold section, a second sample and hold section 
for sampling and holding the output of the first line mem- 
ory for one line, a second line memory for storing one 
line of data from the second sample and hold section, a 
first frequency divider for sending a timing signal, 45 
obtained by dividing the horizontal dot clock by 8, to the 
first sample and hold section, and a second frequency 
divider for sending a timing signal, obtained by dividing 
the vertical sync signal by 8, to the second sample and 
hold section. so 

At the first timing, as shown in FIG. 58 A. the image 
of the teacher saying "ELEVEN" as the main picture and 
the superimposition "ELEVEN" as the sub-picture which 
has been turned to a mosaic pattern by the DSP 65 are 
superimposed on the monitor section 6. and the 55 
teacher's voice "ELEVEN" is reproduced from the 
speaker section 8. At the next timing, as shown in FIG. 
58B. the image of the teacher saying "GOOD" ard the 
superimposition "ELEVEN" as the sub-picture are 



superimposed on the monitor section 6 and the 
teacher's voice "GCXDD" is reproduced from the speaker 
8. 

The reproduction of the no-sound period between 
phrases may be skipped. 

A special reproduction, such as a strobe effect or 
afterimage effect, may be accomplished by performing 
predetermined data processing on the main picture 
data, delayed in the above-described manner, and com- 
bining the processed main picture data with main pic- 
ture with the zero delay time. 

Next, the video data in the logic formats shown in 
FIGS. 4 to 31. a method of recording data on the optical 
disk 10 to reproduce the video data, and a recording 
system to which the recording method is applied will be 
explained with reference to FIGS. 59 to 64. 

FIG. 59 shows an encoder system that creates a 
video file of a title set 84 in which the video data is 
encoded. In the system of FIG. 59. for example, a vide- 
otape recorder (VTR) 201 . an audiotape recorder (ATR) 
202. and a sub-picture source 203 are used as sources 
of the main video data, the audio data, and the sup-pic- 
ture data. Under the control of a system controller (Sys 
con) 205. they create the main video data, audio data, 
and sup-picture data, which are supplied to a video 
encoder (VENC) 206. an audio encoder (AENG) 207. 
and a sub-picture encoder (SPENG) 208. respectively. 
Under the control of the system controller (Sys con) 
205, these encoders 206. 207, and 208 perform A/D 
conversion of the main video data, audio data, and sup- 
picture data and encode them by the respective com- 
pression schemes. The encoded main video data, audio 
data, and sub-picture data (Gomp Video, Comp Audio. 
Comp Sub-pict) are stored in memories 210. 211, and 
212. The main video data, audio data, and sub-picture 
data (Comp Video. Comp Audio. Comp Sub-pict) are 
outputted to a file formatter (FFMT) 214 under the con- 
trol of the system controller (Sys con) 205. which con- 
verts them so that they may have a file structure of video 
data for the system as explained earlier. Then, under 
the control of the system controller (Sys con) 205, the 
setting conditions for each data item and the manage- 
ment information including attributes are stored in a 
memory 216 in the form of files. 

Explained next will be a standard flow of an encod- 
ing process in the system controller (Sys con) 205 that 
creates a file from video data. 

According to the flow of FIG. 60, the main video 
data and the audio data are encoded and the encoded 
main video data and audio data (Comp Video, Con^ 
Audio) are supplied. Specifically, when the encoding 
process is started, as shown in step S70 of FIG. 60, the 
parameters necessary for encoding the main video data 
and audio data are set. Part of the set parameters are 
stored in the system controller (Sys con) 205 and at the 
same time, are used at the file formatter (FFMT) 214. As 
shown in step S271 , the main video data is pre-encoded 
using the parameters and the optimum distribution of 
the amount of codes is calculated. Then, on the basis of 



23 

BNSDOCID: <EP 0730272A2J_> 



45 



EP 0 730 272 A2 



46 



the code amount distribution obtained in the pre-encod- 
ing. the main video data is encoded as shown in step 
S272. At the same time, the audio data is also encoded 
at step S272. As shown in step in S273. if necessary, 
the main video data is partially encoded again and the 5 
reencoded portion of the main video data is replaced 
with the old one. Through the series of steps, the main 
video data and audio data are encoded. Furthermore, 
as shown in steps S274 and S275. the sub-picture data 
is encoded and the encoded sub-picture data (Comp 10 
Sub-pict) is supplied. Namely, the parameters neces- 
sary for encoding the sub-picture data Is set. As shown 
in step S274, part of the parameters are stored in the 
system controller (Sys con) 205 and used in the file for- 
matter (FFMT) 214. On the basis of the parameters, the ys 
sub-picture data is encoded. By the process, the sup- 
picture data is encoded. 

According to the flow of FIG. 61 , the encoded main 
video data, audio data, and sup-picture data (Com 
Video, Com Audio, Comp Sub-pict) are mixed and con- 20 
verted so as to form a video data title set structure as 
explained in FIGS. 4 and 12. Specifically, as shown in 
step S276, a cell is set as the smallest unit of the video 
data and cell playback information on a cell (C_PBI) is 
created. Then, as shown In step S277, the structure of 25 
the cells constituting a program chain and the main 
video, sub-picture, and audio attributes (the information 
obtained in encoding the respective data items are used 
part of these attributes) are set and the video title set 
information management table information (VTSI_MAT) 30 
98 containing information on a program chain and a 
video title set time search map table (VTS_MAPT) 142 
are created. At this time, as the need arises, a video title 
set direct access pointer table (VTS_DAPT) is also cre- 
ated. The encoded main video data, audio data, and 35 
sup-picture data (Com Video, Comp Audio. Comp Sup- 
pict) are subdivided into specific packs. An NV pack is 
placed at the head of each VOBU unit so that playback 
can be effected In the order of time code of each data 
item. With the NV packs arranged this way, each data 40 
cell is positioned so that a video object (VOB) may be 
conposed of a plurality of cells as shown in FIG. 6. A 
set of such video objects is formatted into the title set 
structure. 

Time stamps like PTS and DTS are described in 45 
each pack. 

According to the first embodiment, described in the 
PTS of a sub-picture, for example, is a time arbitrarily 
delayed from the PTS of main picture data or audio data 
which is to be reproduced in the same time slot as the so 
sub-picture. 

According to the second and other embodiments, 
the same time as described in the PTS of main picture 
data or audio data which is to be reproduced in the 
same time slot is described in the PTS of a sub-picture. 55 

In the flow of FIG. 61, the program chain informa- 
tion (PGI) Is obtained in the process of step S277 by 
using the database in the system controller (Sys con) 
205 or entering data again as the need arises. 



FIG. 62 shows a disk formatter system that records 
on an optical disk the title set formatted as described 
above. In the disk formatter system of FIG. 48, the 
memories 220, 222 in which the created title set is 
stored supply these file data items to a volume formatter 
(VFMT) 226. In the volume formatter (VFMT) 226 
extracts the management information from the title sets 
84. 86, produces a video manager 71, and create the 
logic data to be recorded on the disk 10 in the arrange- 
ment of FIG, 4. A disk formatter (DFMT) 228 adds error 
correction data to the logic data created at the volume 
formatter (VFMT) 226. thereby reconverting the logic 
data into physical data to be recorded on the disk. A 
modulator 230 converts the physical data created at the 
disk formatter (DFMT) 228 into the recording data to be 
recorded actually on the disk. Then, a recorder 232 
records the modulated recording data on the disk 10. 

A standard flow for creating the aforementioned 
disk will be described with reference to FIGS. 63 and 
64. FIG. 63 shows the flow for creating the logic data to 
be recorded on the disk 10. Specifically, as shown in 
step S280. parameter data items, including the number 
of video data files, their arrangement, and the size of 
each video data file, are set first. Next, in step S281 
video manger 71 is generated from the parameters set 
and the video title set information 281 of each video title 
set 72. In step S282. the video manager 71 and the 
video tile set 71 are arranged in the order mentioned, 
according to their logic block numbers, thereby generat- 
ing logic data which is to be recorded on the disk 10. 

Thereafter, the flow for creating the physical data to 
be recorded on the disk as shown in FIG. 64 is exe- 
cuted. Specifically, as shown in step S283, the logic 
data Is divided into units of a specific number of bytes, 
thereby forming error correction data. Next, as shown in 
step S284. the logic data divided into units of a specific 
number of bytes are mixed with the created error correc- 
tion data to form physical sectors. Thereafter, as shown 
in step S285. physical data is created by combining 
physical sectors. In this way. the modulating process 
based on certain rules is performed on the physical data 
created in the flow of FIG, 64, thereby forming the 
recording data. Thereafter, the recording data Is 
recorded on the disk 10. 

The above-described data structure can be applied 
not only to a case where the data is recorded on record- 
ing mediums, such as optical disks, and then the disks 
are distributed to the users, but also to a communication 
system as shown in FIG. 65. Specifically, according to 
the procedure shown in FIGS. 59 to 62, an optical disk 
1 0 in which a video manager 71 and video title set 72 as 
shown in FIG. 4 may be loaded into a reproducing unit 
300. from whose system CPU section 50 the encoded 
data is taken out digitally and transmitted by a modula- 
tor/transmitter 310 to the users or the cable subscribers 
by radio or via a cable. Furthermore, the encoding sys- 
tem 320 shown in FIGS. 59 and 62 may create the data 
encoded on the provider side, such as a broadcasting 
station and the encoded data may be transmitted by the 
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modulator/transmitter 310 to the users or the cable sub- 
scribers by radio or via a cable. In such a communica- 
tion system, the information in the video manager 71 is 
modulated at the modulator/transmitter 310 and then 
supplied to or is directly supplied to the users free of 5 
charge. When a user is interested In the title, the modu- 
lator/transmitter 310 transmits the title set 72 at the 
user's or subscriber's request by radio or via a cable. 
Under the control of the video manager 71, the video 
title set information 94 is first transferred and then the 10 
title video object 95 in the video title set reproduced 
according to the title set information 94 is transferred. At 
this time, if necessary, the video title set menu video 
object 95 is also transmitted. The transferred data is 
received by a receiver/demodulator 400 on the user 75 
side and is processed as encoded data at the system 
CPU section 50 of the reproducing unit on the user or 
subscriber side of FIG. 1 in the same manner as in the 
above-described reproducing process, whereby the 
video data is reproduced. 20 

In transferring the video title set 72, the video object 
sets 95, 96 are transferred using the video object unit 85 
of FIG. 6 as a unit. At the head of the video object unit 
85, an NV pack 86 containing video playback and 
search information is arranged. Furthermore, because 25 
the NV pack contains the addresses of the video object 
units to be reproduced tnnmediately before and after the 
video object unit 85 to which the NV pack 86 belongs, 
even if the video object unit 85 is lost during transfer for 
some reason, the video object can be reproduced relia- 30 
biy on the user side by requesting the lost video object 
unit 85 to be transmitted again. Furthermore, even if 
transfer is not carried out in the order of playback of 
video object units, because the system ROM/FtAM sec- 
tion 52 on the user side holds the accurate playback 35 
information on program chains, the system CPU section 
50 can specify the order of playback referring to the 
address data in its NV pack. 

Claims 40 

1 . A recording medium characterized by connprising: 

a plurality of data units recorded thereon, 
which become objects to be reproduced in a time 
sequential manner, each object being reproduced 45 
within a given time period; 

each of said data units constituted of a 
stream of a plurality of data packs and playback 
time information, said data packs containing pack- 
ets of reproduction data including at least one of so 
audio data, video data and sub-picture data; 

said time information (PTS) of that data unit 
which includes a sub-picture being later by a prede- 
termined time than time information (PTS) of that 
data unit which includes audio data or video data; 55 

reproduction data of said data unit including 
said sub-picture being reproduced after reproduc- 
tion of reproduction data of said data unit including 
said audio data or video data. 



2. A recording medium according to claim 1 , charac- 
terized in that said audio data and sub-picture are 
of a same language. 

3. A recording apparatus for recording reproduction 
data on a recording medium (10), characterized by 
comprising: 

means for generating a plurality of data 
packs and playback time information (PTS). said 
data packs containing packets of reproduction data 
including at least one of audio data, video data and 
sub-picture data; 

means for preparing a plurality of data units 
including playback time information (PTS) and hav- 
ing a stream of data packs to be reproduced within 
a given time period as a unit; and 

means for recording said data units on said 
recording medium (10) in a time-sequentially repro- 
ducible manner, 

time information (PTS) of that data unit 
which includes a sub-picture to be reproduced 
within said given time period being later by a prede- 
termined time than time information (PTS) of that 
data unit which includes audio data or video data to 
be reproduced within said given time period. 

4. A recording apparatus according to claim 3. charac- 
terized in that said audio data and sub-picture are 
of a same language. 

5. A method of recording reproduction data on a 
recording medium (10), characterized by compris- 
ing the steps of: 

generating a plurality of data packs contain- 
ing packets of reproduction data including at least 
one of audio data, video data and sub-picture data; 

preparing a plurality of data units including 
playback time information (PTS) and having a 
stream of data packs to be reproduced within a 
given time period as a unit; and - 

recording said data units on said recording 
medium (10) in a time-sequentially reproducible 
manner, 

time information (PTS) of that data unit 
which includes a sub-picture to be reproduced 
within said given time period being later by a prede- 
termined time than time information (PTS) of that 
data unit which includes audio data or video data to 
be reproduced within said given time period. 

6. A method according to daim 5. characterized in 
that said audio data and sub-picture are of a same 
language. 

7. A reproducing apparatus for reproducing reproduc- 
tion data from a recording medium (10), character- 
ized by comprising: 

means (30, 50, 52, 54, 56) for acquiring, 
from said recording medium (10). a plurality of data 
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units recorded thereon, which become objects to be 
reproduced in a time sequential manner, each 
object being reproduced within a given time period, 
each of said data units being constituted of a 
stream of a plurality of data packs and playback 5 
time information, said data packs containing pack- 
ets of reproduction data including at least one of 
audio data, video data and sub-picture data, 

time Information (PTS) of that data unit 
which Includes a sub-picture to be reproduced 10 
within a given time period being later by a predeter- 
mined time than time information (PTS) of that data 
unit which includes audio data or video data to be 
reproduced within said given time period; and 

means for converting said data unit of said 15 
acquired reproduction data to a reproduction signal 
in accordance with said time information (PTS) 
included in said data unit and reproducing sard 
reproduction signal, 

whereby after reproduction data of that data 20 
unit which includes audio data or video data to be 
reproduced within a given time period is repro- 
duced, said reproducing means reproduces repro- 
duction data of said data unit including said sub- 
picture to be reproduced within said given period. 25 



8. A reproducing apparatus according to claim 7. 
characterized in that after reproduction data of said 
data unit including audio data and reproduction 
data of said data unit including video data are 30 
simultaneously reproduced, reproduction data of 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
including video data are simultaneously reproduced 35 
again after reproduction of said reproduction data 

of said data unit including said sub-picture is com- 
pleted. 

9. A reproducing apparatus according to claim 7, 40 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 45 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, so 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

55 

10. A reproducing apparatus according to claim 7, 
characterized in that while reproduction data of said 
data unit including video data is being reproduced 
at a predetermined point of time, reproduction data 



of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, r^roduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data, both data being 
associated with that point of time, are simultane- 
ously reproduced. 

11. A reproducing apparatus according to claim 7. 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

12. A reproducing apparatus according to claim 7. 
characterized in that said audio data and said sub- 
picture are of a same language. 

13. A method of reproducing reproduction data from a 
recording medium (10), characterized by compris- 
ing the steps of: 

acquiring (30, 50. 52. 54. 56), from said 
recording medium (10), a plurality of data units 
recorded thereon, which become objects to be 
reproduced in a time sequential manner, each 
object being reproduced within a given time period, 
each of said data units being constituted of a 
stream of a plurality of data packs and playt>ack 
time information (PTS), said data packs containing 
packets of reproduction data including at least one 
of audio data, video data and sub-picture data, 

time information (PTS) of that data unit 
which includes a sub-picture to be reproduced 
within a given time period being later by a predeter- 
mined time than time information (PTS) of that data 
unit which includes audio data or video data to be 
reproduced within said given time period; and 

converting said data unit of said reproduc- 
tion data to a reproduction signal in accordance 
with said time information (PTS) included in said 
data unit and reproducing said reproduction signal. 

whereby after reproduction data of that data 
unit which includes audio data or video data to be 
reproduced within a given time period is repro- 
duced, reproduction data of said data unit including 
said sub-picture to be reproduced within said given 
period is reproduced. 
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14. A reproducing method according to claim 13. char- 
acterized In that after reproduction data of said data 
unit inclixling audio data and reproduction data of 
said data unit including video data are simultane- 
ously reproduced, reproduction data of said data 
unit Including a sub-picture is reproduced, and 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data are simultaneously reproduced 
again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 

15. A reproducing method according to claim 13, char- 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simuKaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

16. A reproducing method according to claim 13, char- 
acterized in that while reproduction data of said 
data unit including video data is being reproduced 
at a predetermined point of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit Including video data, both data being 
associated with that point of time, are simultane- 
ously reproduced. 

17. A reproducing method according to claim 13. char- 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data Is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit Including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 



18. A reproducing method according to claim 13. char- 
acterized in that said audio data and said sub-pic- 
ture are of a same language. 

5 19. A reproducing apparatus for reproducing reproduc- 
tion data from a recording medium (10), character- 
ized by comprising: 

means (30, 50. 52, 54. 56) for acquiring, 
from said recording medium (10), a plurality of data 

/ 0 units recorded thereon , which become obj ects to be 
reproduced in a time sequential manner, each 
object being reproduced within a given time period, 
each of said data units being constituted of a 
stream of a plurality of data packs and playt^ack 

15 time information (PTS). said data packs containing 
packets of reproduction data including at least one 
of audio data, video data and sub-picture data; and 
means for converting said data unit of said 
acquired reproduction data to a reproduction signal 

20 in accordance with said time information (PTS) 
Included in said data unit and reproducing said 
reproduction signal, 

whereby after reproduction data of that data 
unit which includes audio data or video data to be 

25 reproduced within a given time period is repro- 
duced, said reproducing means reproduces repro- 
duction data of said data unit including said sub- 
picture to be reproduced witiiin said given period 
when a predetermined time previously set elapses. 

30 

20. A reproducing apparatus according to claim 19, 
characterized in that after reproduction data of said 
data unit including audio data and reproduction 
data of said data unit including video data are 

35 simultaneously reproduced, reproduction data of 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
Including video data are simultaneously reproduced 

40 again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 

21. A reproducing apparatus according to claim 19. 
45 characterized in that when a predetermined time 

elapses after reproduction data of said data unit 
Including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 

50 including a sub-picture is reproduced while said 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, 

55 reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with tiiat 
point of time, are simultaneously reproduced. 
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22. A reproducing apparatus according to claim 19, 
characterized in that while reproduction data of said 
data unit including video data is being reproduced 
at a predetermined point of time, reproduction data 

of said data unit including a sub-picture is repro- 5 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data, both data being 10 
associated with that point of time, are simultane- 
ously reproduced. 

23. A reproducing apparatus according to claim 19, 
characterized in that when a predetermined time 15 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 20 
duction data of said data unit including video data at 
that point of time is kept reproduced, arxi when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit including audio data 25 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

24. A reproducing apparatus according to claim 19, 30 
characterized in that said audio data and said sub- 
picture are of a same language. 

25. A method of reproducing reproduction data from a 
recording medium (10), characterized by compris- 35 
ing the steps of: 

acquiring (30, 50, 52, 54. 56), from said 
recording medium (10). a plurality of data units 
recorded thereon, which become objects to be 
reproduced in a time sequential manner, each 40 
object being reproduced within a given time period, 
each of said data units being constituted of a 
stream of a plurality of data packs and playback 
lime information (PTS), said data packs containing 
packets of reproduction data including at least one 4S 
of audio data, video data and sub-picture data; and 

converting (57. 58, 60. 62) said data unit of 
said reproduction data to a reproduction signal in 
accordance with sard time information (PTS) 
included in said data unit and reproducing said so 
reproduction signal. 

whereby after reproduction data of that data 
unit which includes at least one of audio data and 
video data to be reproduced within a given time 
period is reproduced, reproduction data of said 55 
data unit including said sub-picture to be repro- 
duced within said given period is reproduced when 
a predetermined time previously set elapses. 



26. A reproducing method according to claim 25, char- 
acterized in that after reproduction data of said data 
unit including audio data and reproduction data of 
said data unit including video data are simultane- 
ously reproduced, reproduction data of sard data 
unit including a sub-picture is reproduced, and 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data are simultaneously reproduced 
again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 

27. A reproducing method according to claim 25. char- 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of sard 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of sard 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

28. A reproducing method according to claim 25. char- 
acterized in that while reproduction data of said 
data unit including video data is being reproduced 
at a predetermined point of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of sard 
data unit including video data, both data being 
associated with that point of time, are simultane- 
ously reproduced. 

29. A reproducing method according to claim 25, char- 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while sard repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 
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30. A reproducing method according to claim 25, char- 
acterized in that said audio data and said sub-pic- 
ture are of a same language. 

31 . A reproducing apparatus for reproducing reproduc- 
tion data from a recording medium (10), character- 
ized by comprising: 

means (30, 50. 52. 54, 56) for acquiring, 
from said recording medium (10), a plurality of data 
units recorded thereon, which become objects to be 
reproduced in a time sequential manner, each 
object being reproduced within a given time period, 
each of said data units being constituted of a 
stream of a plurality of data packs and playback 
time information (PTS), said data packs containing 
packets of reproduction data including at least one 
of audio data, video data and sub-picture data; 

means (4) for setting a delay time (Td) at 
which reproduction data of that data unit which 
includes a sub-picture to be reproduced within a 
given time period after reproduction data of that 
data unit which includes audio data or video data to 
be reproduced within said given time period is 
reproduced; and 

means (57, 58. 60, 62) for converting said 
data unit of said acquired reproduction data to a 
reproduction signal in accordance with said time 
information (PTS) included in said data unit and 
reproducing said reproduction signal, 

whereby after reproduction data of that data 
unit which includes at least one of audio data and 
video data to be reproduced within a given time 
period is reproduced, said reproducing means 
reproduces reproduction data of said data unit 
including said sub-picture to be reproduced within 
said given period when said set delay time (Td) 
elapses. 

32. A reproducing apparatus according to claim 31. 
characterized in that after reproduction data of said 
data unit Including audio data and reproduction 
data of said data unit including video data are 
simultaneously reproduced, reproduction data of 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
including video data are simultaneously reproduced 
again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 

33. A reproducing apparatus according to claim 31. 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 



data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 
5 data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

34. A reproducing apparatus according to claim 31. 
10 characterized in that while reproduction data of said 

data unit including video data is being reproduced 
at a predetermined point of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 

15 data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data, both data being 
associated with that point of time, are simultane- 

20 ously reproduced. 

35. A reproducing apparatus according to claim 31, 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 

25 including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 

30 that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 

35 video data, both data being associated with that 
point of time, are simultaneously reproduced. 

36. A reproducing apparatus according to claim 31, 
characterized in that sard audio data and said sub- 

40 picture are of a same language. 

37. A reproducing apparatus according to claim 31, 
characterized in that said sut>-picture is displayed 
as if hidden by a mosaic pattern or black filling for a 

45 delay period. 

38. A method of reproducing reproduction data from a 
recording medium (10), characterized by compris- 
ing the steps of: 

50 acquiring (30, 50, 52, 54, 56). from said 

recording medium (10), a plurality of data units 
recorded thereon, which become objects to be 
reproduced in a time sequential manner, each 
object being reproduced within a given time period, 

55 each of said data units being constituted of a 
stream of a plurality of data packs and playt>ack 
time information (PTS). said data packs containing 
packets of reproduction data including at least one 
of audio data, video data and sub-picture data; 
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setting (4) a delay time (Td) at which repro- 
duction data of that data unit which includes a sub- 
picture to be reproduced within a given time period 
after reproduction data of that data unit which 
includes audio data or video data to be reproduced 5 
within said given time period is reproduced: and 

converting (57, 58. 60. 62) said data unit of 
said reproduction data to a reproduction signal in 
accordance with said time information (PTS) 
included in said data unit and reproducing said 10 
reproduction signal, 

whereby after reproduction data of that data 
unit which includes at least one of audio data and 
video data to be reproduced within a given time 
period is reproduced, reproduction data of said 75 
data unit including said sub-picture to be repro- 
duced within said given period is reproduced when 
said set delay time (Td) elapses. 



39. A reproducing method according to claim 38, char- 20 
acterized in that after reproduction data of said data 
unit including audio data and reproduction data of 
said data unit including video data are simultane- 
ously reproduced, reproduction data of said data 
unit including a sub-picture is reproduced, and 25 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data are simultaneously reproduced 
again after reproduction of said reproduction data 

of said data unit including said sub-picture is com- 30 
pleted. 

40. A reproducing method according to claim 38. char- 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 35 
including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 40 
data at that point of time is Kept reproduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is conrpleted. 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 45 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

41 . A reproducing method according to claim 38, char- 
acterized in that while reproduction data of said so 
data unit including video data is being reproduced 

at a predetermined point of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 55 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data, both data being 



associated with that point of time, are simultane- 
ously reproduced. 

42. A reproducing method according to claim 38, char- 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

43. A reproducing method according to claim 38. char- 
acterized in that said audio data and said sub-pic- 
ture are of a same language. 

44. A reproducing method according to claim 38, char- 
acterized in that said sub-picture is displayed as if 
hidden by a mosaic pattern or black filling for a 
delay period. 

45. A reproducing apparatus for reproducing reproduc- 
tion data from a recording medium (10), character- 
ized by comprising: 

means (30, 50, 52, 54, 56) for acquiring, 
from said recording medium (10), a plurality of data 
units recorded thereon, which become objects to be 
reproduced in a time sequential manner, each 
object being reproduced within a given time period, 
each of said data units being constituted of a 
stream of a plurality of data packs and playb>ack 
time information (PTS). said data packs containing 
packets of reproduction data including at least one 
of audio data, video data and sub-picture data, 

said time information (PTS) of that data unit 
which includes a sub-picture consisting of a play- 
back start time (PTS) and playback stop time 
(PTS): 

means (57) for computing a delay time of 
said data unit including said sub-picture from said 
playt>ack start time (PTS) and playback stop time 
(PTS) recorded in said data unit including said sut>- 
picture of said acquired reproduction data; and 

means (57, 58. 60, 62) for converting said 
data unit of said acquired reproduction data to a 
reproduction signal in accordance with said time 
information (PTS) included in said data unit and 
reproducing said reproduction signal. 

whereby after reproduction data of that data 
unit which includes at least one of audio data and 
video data to be reproduced within a given time 
period is reproduced, said reproducing means 
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reproduces reproduction data of said data unit 
including said sub-picture to be reproduced within 
said given period when said computed delay time 
elapses. 

46. A reproducing apparatus according to daim 45. 
characterized in that after reproduction data of said 
data unit including audio data and reproduction 
data of said data unit including video data are 
simultaneously reproduced, reproduction data of 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
including video data are simultaneously reproduced 
again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 

47. A reproducing apparatus according to claim 45. 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced white said 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said 
data unit irrcluding said sub-picture is completed, 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

48. A reproducing apparatus according to claim 45, 
characterized in that while reproduction data of said 
data unit including video data is being reproduced 
at a predetermined point of time, reproduction data 
of said data unit Including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data. tx>th data being 
associated with that point of time, are simultane- 
ously reproduced. 

49. A reproducing apparatus according to daim 45. 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit induding audio data 



and reproduction data of said data unit induding 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

5 50. A reproducing apparatus according to claim 45. 
characterized in that said audio data and said sub- 
picture are of a same language. 

51. A reproducing apparatus according to claim 45, 
10 characterized in that said sub-picture is displayed 

as if hidden by a mosaic pattern or black filling for a 
delay period. 

52. A method of reproducing reproduction data from a 
15 recording medium (10), characterized by compris- 
ing the steps of: 

acquiring (30, 50. 52. 54, 56), from said 
recording medium (10). a plurality of data units 
recorded thereon, which become objects to be 

20 reproduced in a time sequential manner, each 
object being reproduced within a given time period, 
each of said data units being constituted of a 
stream of a plurality of data packs and playt>ack 
time information (PTS). said data packs containing 

25 packets of reproduction data induding at least one 
of audio data, video data and sub-picture data, 

computing (57) a delay time (Td) at which 
reproduction data of said data unit including said 
sub-picture from a playback start time (PTS) and 

30 playback stop time (PTS) recorded in said data unit 
including said sub-picture of acquired reproduction 
data: and 

converting (57. 58, 60. 62) said data unit of 
said reproduction data to a reproduction signal in 

35 accordance with said time information (PTS) 
included in said data unit and reproducing said 
reproduction signal, 

whereby after reproduction data of that data 
unit which includes at least one of audio data and 

40 video data to be reproduced within a given time 
period is reproduced, reproduction data of said 
data unit including said sub-picture to be repro- 
duced within said given period is reproduced when 
said computed delay time elapses. 

45 

53. A reproducing method according to claim 52, char- 
acterized in that after r^roduction data of said data 
unit including audio data and reproduction data of 
said data unit including video data are simultane- 

50 ousiy reproduced, reproduction data of said data 
unit induding a sub-picture is reproduced, and 
reproduction data of said data unit induding audio 
data and reproduction data of said data unit includ- 
ing video data are simultaneously reproduced 

55 again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 
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54. A reproducing method according to claim 52. char- 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simultaneously 5 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
r^roduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said w 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 15 

55. A reproducing method according to claim 52. char- 
acterized in that while reproduction data of said 
data unit including video data is being reproduced 

at a predetermined point of time, reproduction data 20 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 25 
data unit including video data, both data being 
associated with that point of time, are simultane- 
ously reproduced. 

56. A reproducing method according to claim 52. char- 30 
acterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 35 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 40 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

45 

57. A reproducing method according to claim 52. char- 
acterized in that said audio data and said sub-pic- 
ture are of a same language. 

58. A reproducing method according to claim 52, char- so 
acterized in that said sub-picture is displayed as if 
hidden by a mosaic pattern or black filling for a 
delay period. 

59. A reproducing apparatus characterized by compris- ss 
ing: 

reading means (30, 50, 52, 54, 56) for read- 
ing a main picture data unit consisting of a stream 
of packs of at least one main picture data for repro- 



ducing a main picture, an audio data unit consisting 
of a stream of packs of at least one audio data 
reproducible in a same time slot as said main pic- 
ture data unit, and a sub-picture data unit consisting 
of a stream of packs of at least one sub-picture data 
reproducible after a predetermined time after repro- 
duction of at least one of said main picture data unit 
and said audio data unit; 

first control means (57, 58, 60) for converting 
at least one of said main picture data unit and audio 
data unit, read by said reading means (30, 50. 52, 
54. 56), to a reproduction output; and 

second control means (57. 62) for converting 
said sub-picture data unit, read by said reading 
means (30, 50, 52, 54, 56), to a reproduction output 
after conversion of at least one of said main picture 
data unit and audio data unit to said reproduction 
output by said first control means (57, 58. 60). 

60. A reproducing apparatus according to claim 59, 
characterized in that after reproduction data of said 
data unit including audio data and reproduction 
data of said data unit including video data are 
simultaneously reproduced, reproduction data of 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
including video data are simultaneously reproduced 
again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 

61. A reproducing apparatus according to claim 59, 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

62. A reproducing apparatus according to claim 59. 
characterized in that while reproduction data of said 
data unit including video data is being reproduced 
at a predetermined poirit of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data, both data being 
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associated with that point of time, are simultane- 
ously reproduced. 

63. A reproducing apparatus according to claim 59. 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

64- A reproducing apparatus according to claim 59, 
characterized in that said audio data and said sub- 
picture are of a same language. 

65. A reproducing apparatus characterized by compris- 
ing: 

reading means (30. 50. 52. 54. 56) for read- 
ing a main picture data unit consisting of a stream 
of packs of at least one main picture data for repro- 
ducing a main picture, an audio data unit consisting 
of a stream of packs of at least one audio data 
reproducible in a same time slot as said main pic- 
ture data unit, and a sub-picture data unit consisting 
of a stream of packs of at least one sub-picture data 
reproducible in said same time slot as said main 
picture data unit; 

first control means (57, 58, 60) for converting 
at least one of said main picture data unit and audio 
data unit, read by said reading means (30. 50. 52, 
54, 56). to a reproduction output: and 

second control means (57, 62) for converting 
said sub-picture data unit, read by said reading 
means (30, 50, 52. 54. 56), to a reproduction output 
when a predetermined time previously set elapses 
after conversion of at least one of said main picture 
data unit and audio data unit to said reproduction 
output by said first control means (57. 58. 60). 

66. A reproducing apparatus according to claim 65, 
characterized in that after reproduction data of said 
data unit including audio data and reproduction 
data of said data unit including video data are 
simultaneously reproduced, reproduction data of 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
including video data are simultaneously reproduced 
again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 



67. A reproducing apparatus according to claim 65. 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 

5 data unit including video data are simultaneously 

reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 

ro when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 

/5 point of time, are simultaneously reproduced. 

68. A reproducing apparatus according to claim 65. 
characterized in that while reproduction data of said 
data unit including video data is being reproduced 

20 at a predetermined point of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
25 including audio data and reproduction data of said 
data unit including video data, both data being 
associated with that point of time, are simultane- 
ously reproduced. 

30 69. A reproducing apparatus according to claim 65, 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 

35 reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept r^roduced, and when 
reproduction of said reproduction data of said data 

40 unit including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

45 

70. A reproducing apparatus according to claim 65, 
characterized in that said audio data and said sub- 
picture are of a same language. 

50 71 . A reproducing apparatus characterized by compris- 
ing: 

reading means (30, 50. 52, 54. 56) for read- 
ing a main picture data unit consisting of a stream 
of packs of at least one main picture data for repro- 
55 ducing a main picture, an audio data unit consisting 
of a stream of packs of at least one audio data 
reproducible in a same time slot as said main pic- 
ture data unit, and a sub-picture data unit consisting 
of a stream of packs of at least one sub-picture data 
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reproducible after a predetermined time after repro- 
duction of at least one of said main picture data unit 
and said audio data unit; 

setting means (4) for setting a delay time 
(Td) at which said sub-picture data unit is repro- 5 
duced after reproduction of at least one of said 
main picture data unit and audio data unit; 

first control means (57, 58, 60) for converting 
at least one of said main picture data unit and audio 
data unit, read by said reading means (30, 50, 52, 70 
54, 56). to a reproduction output; and 

second control means (57, 62) for converting 
said sub-picture data unit, read by said reading 
means (30, 50, 52. 54, 56), to a reproduction output 
when said delay time (Td) set by said setting means 15 
(4) elapses after conversion of at least one of said 
main picture data unit and audio data unit to said 
reproduction output by said first control means (57, 
62). 

20 

72. A reproducing apparatus according to claim 71, 
characterized in that after reproduction data of said 
data unit including audio data and reproduction 
data of said data unit including video data are 
simultaneously reproduced, reproduction data of 25 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
including video data are simultaneously reproduced 
again after reproduction of said reproduction data 30 
of said data unit including said sub-picture is com- 
pleted. 

73. A reproducing apparatus according to claim 71, 
characterized in that when a predetermined time 35 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 40 
reproduction data of said data unit including video 
data at that point of time is kept reproduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 45 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

74. A reproducing apparatus according to claim 71, so 
characterized in that while reproduction data of said 
data unit including video data is being reproduced 

at a predetermined point of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction ss 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data, both data being 



associated with that point of time, are simultane- 
ously reproduced. 

75. A reproducing apparatus according to claim 71, 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sub-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
tiiat point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is conrpleted. repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 
video data, both data being associated with that 
point of time, are simultaneously reproduced. 

76. A reproducing apparatus according to claim 71. 
characterized in that said audio data and said sub- 
picture are of a same language. 

77. A reproducing apparatus according to claim 71, 
characterized in that said sub-picture is displayed 
as if hidden by a mosaic pattern or black filling for a 
delay period. 

78. A reproducing apparatus characterized by compris- 
ing: 

reading means (30, 50, 52. 54, 56) for read- 
ing a main picture data unit consisting of a stream 
of packs of at least one main picture data for repro- 
ducing a main picture, an audio data unit consisting 
of a stream of packs of at least one audio data 
reproducible in a same time slot as said main pic- 
ture data unit, and a sub-picture data unit consisting 
of a stream of packs of at least one sub-picture data 
reproducible after a predetermined time after repro- 
duction of at least one of said main picture data unit 
and said audio data unit, a playback start time 
(PTS) and playback stop time (PTS) being recorded 
in said sub-picture data unit; 

means (57) for computing a delay time (Td) 
of said sub-picture data unit from said playback 
start time (PTS) and playback stop time (PTS) in 
said sub-picture data unit, read by said reading 
means (30, 50. 52. 54. 56); 

first control means (57, 58, 60) for converting 
at least one of said main picture data unit and audio 
data unit, read by said reading means (30, 50, 52. 
54, 56), to a r^roductidn output; and 

second control means (57, 62) for converting 
said sub-picture data unit, read by said reading 
means (30, 50. 52, 54, 56), to a reproduction output 
when said delay time (Td) computed by said com- 
puting means (57) elapses after conversion of at 
least one of said main picture data unit and audio 
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data unit to said reproduction output by said first 
control means (57, 62). 

79. A reproducing apparatus according to claim 78. 
characterized in that after reproduction data of said 
data unit including audio data and reproduction 
data of said data unit including video data are 
simultaneously reproduced, reproduction data of 
said data unit including a sub-picture is reproduced, 
and reproduction data of said data unit including 
audio data and reproduction data of said data unit 
including video data are simultaneously reproduced 
again after reproduction of said reproduction data 
of said data unit including said sub-picture is com- 
pleted. 

80. A reproducing apparatus according to claim 78, 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data are simultaneously 
reproduced, reproduction data of said data unit 
including a sub-picture is reproduced while said 
reproduction data of said data unit including video 
data at that point of time is kept r^roduced, and 
when reproduction of said reproduction data of said 
data unit including said sub-picture is completed, 
reproduction data of said data unit including audio 
data and reproduction data of said data unit includ- 
ing video data, both data being associated with that 
point of time, are simultaneously reproduced. 

81. A reproducing apparatus according to daim 78, 
characterized in that while reproduction data of said 
data unit including video data is being reproduced 
at a predetermined point of time, reproduction data 
of said data unit including a sub-picture is repro- 
duced, and when reproduction of said reproduction 
data of said data unit including said sub-picture is 
completed, reproduction data of said data unit 
including audio data and reproduction data of said 
data unit including video data, both data being 
associated with that point of time, are simultane- 
ously reproduced. 

82. A reproducing apparatus according to claim 78, 
characterized in that when a predetermined time 
elapses after reproduction data of said data unit 
including video data and reproduction data of said 
data unit including a sut>-picture are simultaneously 
reproduced, reproduction data of said data unit 
including audio data is reproduced while said repro- 
duction data of said data unit including video data at 
that point of time is kept reproduced, and when 
reproduction of said reproduction data of said data 
unit including said audio data is completed, repro- 
duction data of said data unit including audio data 
and reproduction data of said data unit including 



video data, both data being associated with that 
point of time, are simultaneously reproduced. 

83. A reproducing apparatus according to claim 78. 
5 characterized in that said audio data and said sub- 
picture are of a same language. 

84. A reproducing apparatus according to claim 78, 
characterized in tiiat said sub-picture is displayed 

10 as if hidden by a mosaic pattern or black filling for a 
delay period. 
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